4.C.2
Reporting evaluation results people can understand and use
Énoncé de compétence 4.C.2 d’ASPPB : Communicate evaluation results to stakeholders in an easily understandable manner
Results help only if the people who decide can understand and trust them. Good reporting leads with the answer to the audience's question, uses plain language and formats suited to that audience, states uncertainty honestly and presents effects in absolute terms that are not inflated. Unwelcome findings are reported as fully as welcome ones, and results go back to those who provided the data.
Notions clés
- Relevance and utility
- CDC's 2024 standards call for findings that are actionable, available in time for use and presented in ways that are understandable and culturally responsive, in formats such as briefs, dashboards and discussions of results.
- Absolute over relative
- In a Cochrane review, relative risk reductions were perceived as larger and more persuasive than absolute reductions, and natural frequencies (such as 5 in 100) were understood better than probabilities.
- Uncertainty
- Reporting an estimate with its confidence interval shows how precise it is. A nonsignificant result in a small study is not evidence of no effect; absence of evidence is not evidence of absence.
- Transparency
- Reports should not be tailored to generate particular findings. Discussing interim findings along the way keeps results from surprising stakeholders, and sharing results with those who provided data is part of ethical practice.
- Audience fit
- A board, program staff, funders and participants need different levels of detail. The core findings stay the same; the length, language and format change.
Exemple commenté
A psychologist has evaluated a smoking cessation group offered to clients of an outpatient mental health clinic. At six months, 15 of 100 group members and 10 of 100 clients on the waiting list had quit. The director wants the summary to say the group raised quitting by 50 percent.
- Present the result in absolute terms first: 5 more people quit for every 100 who took part, 15 against 10.
- Add the confidence interval and say plainly whether the study is large enough to rule out a smaller benefit or none.
- If the relative figure appears, place it beside the absolute numbers so readers are not misled about the size of the effect.
- Lead the summary with what the result means for the clinic's decision, and offer group members a short summary of the findings.
Report effects in absolute numbers with their uncertainty, in language matched to the decision the audience faces.
Vérifiez vos acquis
A program reduced hospital readmissions from 4 in every 100 clients to 2 in every 100. Which statement conveys the effect most accurately to a board without statistical training?
Sources
- Kidder and colleagues, CDC Evaluation Framework Work Group (2024), CDC Program Evaluation Framework, 2024, MMWR Recommendations and Reports, 73(6), 1-37 (PMID 39316770), Standards (relevance and utility; transparency; ethics) and Step 6, Act on Findings, including interim findings and sharing results back with those who provided data
- Akl, Oxman, Herrin, Vist, Terrenato, Sperati, Costiniuk, Blank and Schunemann (2011), Using alternative statistical formats for presenting risks and risk reductions, Cochrane Database of Systematic Reviews, CD006776 (PMID 21412897), Main results and authors' conclusions: natural frequencies better understood than probabilities; relative risk reduction perceived as larger and more persuasive
- Altman and Bland (1995), Absence of evidence is not evidence of absence, BMJ, 311(7003), 485 (PMID 7647644), Whole article (title verified on PubMed; no abstract)
- Cumming (2014), The new statistics: Why and how, Psychological Science, 25(1), 7-29 (PMID 24220629), Estimation with confidence intervals in place of reliance on significance tests
- Yarbrough, Shulha, Hopson and Caruthers (2011), The Program Evaluation Standards, Third Edition, Utility standards, including timely and appropriate communicating and reporting