1.B.2
Analyzing data and interpreting statistics
Énoncé de compétence 1.B.2 d’ASPPB : Data analysis methods (e.g., quantitative, qualitative, mixed) and statistical interpretation (e.g., power, effect size, causation vs. association, clinical vs. statistical significance)
This statement covers choosing and interpreting analyses, quantitative, qualitative and mixed, along with power, effect size, association versus causation, and the gap between statistical and clinical significance. Psychologists read these numbers in journals and use them with clients, for example to judge whether a change on a scale is real and meaningful. Misreading a p value or a correlation leads to wrong conclusions in research and in practice.
Notions clés
- P value
- The probability of a result at least as extreme as the one observed, computed assuming the null hypothesis and every other model assumption are true. It is not the probability that the null hypothesis is true, and it does not measure an effect's size or importance.
- Effect size and confidence interval
- An effect size, such as Cohen's d or a correlation, expresses how large a difference or association is, and a confidence interval shows a plausible range for it. Together they convey strength, direction and precision, whereas a p value addresses only chance.
- Power
- The probability that a study detects an effect of a given size if it exists. It rises with larger samples, larger true effects and a less strict alpha level; Cohen tabled the sample sizes needed for .80 power to detect small, medium and large effects.
- Association versus causation
- A correlation can arise from reverse causation, a third variable or selection, so causal claims need design features, such as random assignment or a strong quasi-experimental comparison, not a statistical association alone.
- Clinical significance
- Jacobson and Truax defined clinically significant change as moving out of the dysfunctional range or into the functional range, paired with a reliable change index showing that the change is larger than measurement error alone would produce.
- Qualitative and mixed analysis
- Qualitative analysis identifies patterns of meaning in words or observations, for example by coding themes, and is judged by the transparency and integrity of its procedures rather than by statistical tests. Mixed methods integrate qualitative and quantitative strands to answer one question.
Exemple commenté
A client's score on a depression questionnaire falls from 30 at intake to 22 after eight sessions. The manual reports a standard deviation of 8 and a test-retest reliability of .84. The client asks whether this means she is really better.
- Find the standard error of measurement: 8 times the square root of (1 minus .84), which is 8 times 0.4, or 3.2 points.
- Find the standard error of the difference between two scores: the square root of 2 times 3.2 squared, about 4.5 points.
- Compute the reliable change index: the 8-point drop divided by 4.5 is about 1.8, below the 1.96 that marks change unlikely to be measurement error at the .05 level.
- Interpret for the client: the improvement is encouraging but not yet larger than the scale's error, so continue treatment and keep measuring rather than declaring either recovery or failure.
A change on a scale counts as reliable only when it is larger than what measurement error alone could produce.
Vérifiez vos acquis
A trial reports that a new group program reduced loneliness more than usual care, p = .02. Which interpretation is correct?
Sources
- Greenland, Senn, Rothman, Carlin, Poole, Goodman and Altman (2016), Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations, European Journal of Epidemiology (PMID 27209009), Abstract and the list of 25 misinterpretations
- Grimes and Schulz (2002), An overview of clinical research: the lay of the land, The Lancet (PMID 11809203), Abstract: confidence intervals versus p values
- Cohen (1992), A power primer, Psychological Bulletin (PMID 19565683), Abstract and the table of sample sizes for .80 power
- Shadish, Cook and Campbell (2002), Experimental and Quasi-Experimental Designs for Generalized Causal Inference, Chapter 1: causal inference and its requirements
- Jacobson and Truax (1991), Clinical significance: a statistical approach to defining meaningful change in psychotherapy research, Journal of Consulting and Clinical Psychology (PMID 2002127), Abstract; reliable change index formula and 1.96 criterion in the text
- Levitt, Bamberg, Creswell, Frost, Josselson and Suarez-Orozco (2018), Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology, American Psychologist (PMID 29345485), Qualitative and mixed methods standards (analysis and methodological integrity)