3.C.4
Evaluating outcomes: reliable, meaningful and lasting change
Énoncé de compétence 3.C.4 d’ASPPB : Evaluate treatment/intervention outcomes
Evaluating an outcome asks whether the client changed, whether the change is larger than measurement error, whether it is meaningful, and whether it lasts. For an individual, the reliable change index and a clinical cutoff give a twofold test. For a service, outcomes compared with benchmarks from published trials show how routine care measures up. Single-case designs can show that the treatment, rather than time or chance, produced a change.
Notions clés
- Reliable change
- Jacobson and Truax (1991) proposed the reliable change index, which asks whether a client's change is larger than the instrument's measurement error would plausibly produce.
- Clinical significance
- Change that moves a client out of the range of the dysfunctional population or into the range of the functional population; combined with reliable change it forms a twofold criterion (Jacobson and Truax, 1991).
- Outcome categories
- Clients can then be classed as recovered (reliable change and past the cutoff), improved (reliable change without crossing the cutoff), unchanged, or deteriorated (reliable change for the worse) (Jacobson and Truax, 1991).
- Benchmarking a service
- Comparing a setting's effect sizes with those of published trials. One study found psychotherapy for depression in a managed care setting comparable to trial benchmarks (Minami and colleagues, 2008).
- Single-case designs
- Repeated measurement of one client across phases, such as baseline then treatment, or staggered multiple baselines; visual analysis is the most common method of analysis and has the most support in published standards (Smith, 2012).
Exemple commenté
A 41-year-old finishes 16 sessions of therapy for depression. The manual for the clinic's measure states that a change of at least 9 points is reliable and that 20 separates the clinical and nonclinical ranges. Her score fell from 38 at intake to 26, and she has returned to work part time.
- Reliable change: the 12-point drop exceeds the 9 points the manual requires, so it is unlikely to reflect measurement error alone.
- Clinical significance: 26 is still above the cutoff of 20, so in Jacobson and Truax's terms she is improved but not recovered.
- Look beyond the score: her return to work is a functional gain the symptom scale does not capture, and it belongs in the evaluation.
- Plan with her: discuss continuing, changing approach or ending with a relapse prevention plan, and schedule a follow-up measurement to see whether the gains hold.
Judge an outcome on reliability, clinical meaning, functioning and durability, not on a lower score alone.
Vérifiez vos acquis
A psychologist wants a single-case design to show that social skills training, rather than the passage of time, improved a client's skills. Why is an ABAB (withdrawal) design a poor fit?
Sources
- Jacobson and Truax (1991), Clinical significance: a statistical approach to defining meaningful change in psychotherapy research, Journal of Consulting and Clinical Psychology (PMID 2002127), Abstract; definitions, reliable change index and categories
- Minami, Wampold, Serlin, Hamilton, Brown and Kircher (2008), Benchmarking the effectiveness of psychotherapy treatment for adult depression in a managed care environment: a preliminary study, Journal of Consulting and Clinical Psychology (PMID 18229989), Abstract
- Smith (2012), Single-case experimental designs: a systematic review of published research and current standards, Psychological Methods (PMID 22845874), Abstract; analysis methods
- Kazdin (2011), Single-Case Research Designs: Methods for Clinical and Applied Settings, Second Edition, Chapter on ABAB designs and their limits