Study guide · Area D

Experimental Design

13 of the exam's 175 scored questions (7%) · 9 tasks

This area covers how behavior analysts show that an intervention, and not something else, changed behavior: variables, validity and its threats, the logic and features of single-case designs, how they compare with group designs, the main design types, comparative, component and parametric analyses, and choosing and running a design. Questions give a graph or a study description and ask what it shows, what threatens its conclusion, or which design fits a situation.

A study feature written by ExamDragon, not part of the real exam. Each heading is one of the BACB's tasks for this content area (9 of them), in the BACB's words.

D.1Distinguish between dependent and independent variables.

The dependent variable (DV) is the behavior measured to see whether it changes; in applied work it is the target behavior, defined and measured on its relevant dimension.
The independent variable (IV) is the environmental condition the experimenter manipulates (the intervention or treatment) to see whether it changes the DV.
A functional relation is shown when the DV changes reliably and only when the IV is introduced, withdrawn or varied, with other variables held as constant as possible.
Extraneous (confounding) variables are uncontrolled factors other than the IV that could change the DV; good designs hold them constant or show that they cannot account for the results.
Only one IV should change at a time between adjacent phases; changing two at once means any effect cannot be attributed to either.

In practice: A behavior analyst tests whether a visual schedule reduces a child's transition delays. The seconds of delay at each transition are the DV; introducing and removing the visual schedule is the IV. The school's new bell system, introduced the same week, is a possible confound she records.

Where questions turn: The IV is what the experimenter changes; the DV is what is measured. In applied studies the client's behavior is almost always the DV, even when the client's behavior seems to be doing the changing.

Practise D.1

D.2Distinguish between internal and external validity.

Internal validity is the extent to which a study shows that changes in the DV result from the IV and not from uncontrolled variables; it is established by experimental control.
External validity is the degree to which a study's results generalize to other participants, settings, behaviors, implementers and times.
In single-case research, external validity is established mainly by replication: direct replication (same procedure, same or similar participants) and systematic replication (one or more variables deliberately changed).
The two can be in tension: tight laboratory control can strengthen internal validity while narrowing the range of conditions results apply to, and natural settings can do the reverse.
Social validity, a related but separate concept, is the extent to which goals, procedures and outcomes are acceptable and important to the people affected.

In practice: A reversal design shows clearly that token reinforcement increased one child's on-task behavior in one classroom (good internal validity). Whether the same effect would occur for other children, other teachers or other classrooms is a question of external validity, answered by replicating the study across those variables.

Where questions turn: Internal validity asks whether the IV caused the change in this study; external validity asks whether the finding holds beyond it. A large effect says nothing about external validity by itself.

Practise D.2

D.3Identify threats to internal validity (e.g., history, maturation).

History: an event outside the study occurs during it (a new medication, a family move) and could account for the change.
Maturation: changes within the participant over time (growing older, physical development, fatigue within a session) produce the change.
Testing (repeated exposure to measurement): the measurement itself changes behavior, through practice or reactivity.
Instrumentation: the measurement system changes over time, as through observer drift, new observers or changed definitions.
Others: regression toward the mean (extreme scores tend to move toward the average on retest, a reason to wait for stable baselines), attrition (participants drop out), diffusion of treatment (the intervention spreads to baseline conditions), sequence or carryover effects (one condition's effect influences the next) and multiple treatment interference.
Single-case designs address these threats through repeated measurement, stable baselines, and replication of effects at different times so that a coincidental event cannot explain every change.

In practice: Self-injury drops sharply the week a new intervention starts, but the client also began a new antibiotic for an ear infection that week. The change could reflect history (the medical treatment), so the analyst withdraws and reintroduces the intervention to see if the effect replicates.

Where questions turn: History refers to an outside event that coincides with the phase change; maturation refers to changes within the person that happen with the passage of time.

Practise D.3

D.4Identify the defining features of single-case experimental designs (e.g., individuals serve as their own controls, repeated measures, prediction, verification, replication).

Individuals serve as their own controls: each participant's behavior under the intervention is compared with the same participant's behavior without it, so variability between people is not a confound.
Repeated measures: the DV is measured many times in each condition, which shows level, trend and variability and allows decisions to be made from the data as they come in.
Prediction: a stable baseline predicts what behavior would be in the future if nothing changed.
Verification: showing that the baseline level would have continued without the intervention, for example by withdrawing the intervention and seeing behavior return (reversal) or by showing untreated baselines stay unchanged (multiple baseline).
Replication: repeating the effect of the IV within the study (reintroducing it, or introducing it to another tier) increases confidence in the functional relation.
Together prediction, verification and replication make up baseline logic, the reasoning by which single-case designs demonstrate experimental control.

In practice: In an ABAB design, stable baseline data predict continued high disruption; introducing the intervention lowers it; withdrawing it brings disruption back toward baseline (verification); reintroducing it lowers it again (replication).

Where questions turn: Verification is the demonstration that the baseline prediction was correct, not the replication of the intervention effect; replication is producing the effect again.

Practise D.4

D.5Identify the relative strengths of single-case experimental designs and group designs.

Single-case designs show the effects of an IV on individuals, the level at which behavior occurs and at which practitioners make decisions; group means can hide that some participants improved and others got worse.
Single-case designs are flexible: conditions can be changed in response to the data during the study, and they need few participants, which suits rare populations and applied settings.
Single-case designs require repeated, direct measurement and stable data, can take a long time, and establish generality only gradually through replication.
Group designs (randomized comparisons of groups) can estimate average effects in a population, handle many participants efficiently, and suit questions about policy or the average result of a program.
Group designs typically measure each person only a few times, so they show little about within-person variability, and statistical significance does not indicate whether a change matters for any individual.

In practice: A school district wants to know whether a classroom program raises average reading scores across 40 schools, a question suited to a group design. A behavior analyst who needs to know whether a specific intervention reduces one student's aggression uses a single-case design.

Where questions turn: Single-case designs do not lack generality by nature; generality is built through replication, while group designs trade individual-level analysis for population estimates.

Practise D.5

D.6Critique and interpret data from single-case experimental designs.

First ask whether each condition has enough data points, stable enough, to show a level and trend; a single point or a highly variable phase supports no conclusion.
Then judge each phase change: is there a change in level, trend or variability, how immediate is it, and how much do data overlap between conditions?
Experimental control requires the effect to be replicated: a widely used convention (not a fixed rule) is at least three demonstrations of the effect at different points in time (for example, the three phase changes A to B, B to A and A to B in an ABAB design, or three staggered tiers in a multiple baseline).
Look for alternative explanations: baseline trends in the therapeutic direction, changes in tiers that had not yet received the IV (in a multiple baseline), carryover between alternating conditions, or a confound introduced at the same time.
Check the measurement: IOA and procedural integrity reported, the measure sensitive to change, and the scale of the graph not distorting the effect.
Finally, judge social significance: is the change large enough to matter for the person, whatever the experimental control?

In practice: A multiple-baseline graph across three students shows a clear drop in call-outs for each student only after the intervention begins for that student, while the others stay at baseline. The analyst concludes experimental control; had the second student's call-outs dropped when the first student started, she would suspect a confound such as a whole-class effect.

Where questions turn: A change in behavior after one phase change, with no replication, shows a correlation with the intervention, not a functional relation.

Practise D.6

D.7Distinguish among reversal, multiple-baseline, multielement, and changing-criterion designs.

Reversal (ABAB, withdrawal) designs: baseline, intervention, return to baseline, return to intervention. They show strong control, but they need a reversible behavior and may be inappropriate when withdrawing an effective treatment would be harmful or unacceptable. A DRO or noncontingent reversal compares the IV with a control condition rather than a full return to baseline.
Multiple-baseline designs: the IV is introduced in a staggered sequence across behaviors, settings or participants (tiers) while the others remain in baseline. They need no withdrawal, suit irreversible behaviors and harmful-to-withdraw situations, but require tiers that are functionally independent and yet similar enough to respond to the same IV. Multiple probe designs use intermittent baseline probes instead of continuous baseline measurement.
Multielement (alternating treatments) designs: two or more conditions alternate rapidly (by session or within sessions), usually counterbalanced, and are compared through separated data paths. They compare treatments quickly and need no stable baseline, but risk multiple treatment interference and carryover.
Changing-criterion designs: after baseline, the intervention is applied in phases with a stepwise criterion, and control is shown when behavior changes to match each new criterion. They suit behaviors that should change gradually (increasing exercise, reducing cigarettes) but not those that change abruptly.
Choose by question and constraints: reversibility, the ethics of withdrawal, the number of behaviors or settings or people available, and whether the aim is to compare treatments or demonstrate one effect.

In practice: A behavior analyst teaches a client to tie her shoes, a skill that will not reverse. She uses a multiple baseline across three skills (shoes, buttons, zippers), introducing instruction for each in turn. To compare two prompting methods for her vocabulary learning, she uses a multielement design instead.

Where questions turn: A multiple-baseline design does not require withdrawing treatment; a multielement design compares conditions by rapid alternation, not by sequential phases.

Practise D.7

D.8Identify rationales for conducting comparative, component, and parametric analyses.

A comparative analysis compares two or more distinct interventions to find which is more effective or efficient for a given behavior or person (for example, comparing two prompting procedures in a multielement design).
A component analysis identifies which parts of a treatment package are responsible for its effect. In a drop-out analysis the whole package is presented and components are removed one at a time; in an add-in analysis components are assessed individually or in combinations.
A parametric analysis compares different values of the same IV (different reinforcement magnitudes, schedule values, session lengths) to find the most effective or efficient value.
Reasons to conduct them: to choose the best of several options, to remove unnecessary components that cost time or effort, to understand why a package works, and to identify the optimal amount or intensity of an intervention.
Each type answers a different question: which treatment (comparative), which parts (component), how much (parametric).

In practice: A token economy with praise, tokens and response cost reduces disruption. To learn whether response cost is needed, the team removes it while keeping the rest (component analysis). They then test exchange rates of 5, 10 and 20 tokens (parametric analysis).

Where questions turn: Testing different amounts of one intervention is parametric; testing different parts of a package is component; testing different interventions against each other is comparative.

Practise D.8

D.9Apply single-case experimental designs.

Begin with the question: is the aim to show that one intervention works, to compare several, or to find the effective components or values of one?
Collect a stable baseline (or establish that it is stable enough) before introducing the IV; change only one variable at a time between adjacent conditions.
Choose the design that fits the behavior and the ethics: reversal for reversible behaviors where withdrawal is acceptable; multiple baseline or multiple probe for irreversible skills or when withdrawal is unacceptable; multielement to compare treatments quickly; changing criterion for gradual change.
In a multiple baseline, choose independent tiers, start the IV in the next tier only when the previous tier shows a clear change and the remaining baselines are stable.
Keep measurement consistent across conditions, collect IOA and procedural integrity, and decide when to change phases from the data (stability and trend), not from the calendar.
Plan for replication within the study (at least three demonstrations of effect) and across participants when possible.

In practice: A behavior analyst wants to show that a self-monitoring program increases on-task behavior for three students in different classes. Withdrawing a successful program is undesirable, so she uses a multiple baseline across students, starting with the student whose baseline is most stable and moving to the next only after a clear change in the first.

Where questions turn: Introducing the IV in the next tier of a multiple baseline before the previous tier has changed, or while baselines are still trending, undermines the logic of the design.

Practise D.9

Key terms

Dependent variable: The behavior measured to see whether it changes.
Independent variable: The condition the experimenter manipulates to see whether it changes the dependent variable.
Confounding variable: An uncontrolled factor that could explain changes in the dependent variable.
Internal validity: The extent to which changes in the dependent variable result from the independent variable.
External validity: The extent to which results generalize beyond the study's conditions.
Baseline logic: Prediction, verification and replication, the reasoning by which single-case designs show control.
Reversal design: A design that alternates baseline and intervention phases (ABAB) to show control.
Multiple-baseline design: A design that introduces the IV in staggered sequence across behaviors, settings or participants.
Multielement design: A design that rapidly alternates two or more conditions to compare their effects.
Changing-criterion design: A design that shows control by behavior changing to match stepwise criteria.
Component analysis: An analysis that identifies which parts of a treatment package are responsible for its effect.
Parametric analysis: An analysis that compares different values of the same independent variable.
Multiple treatment interference: The effect of one treatment influenced by exposure to another in the same study.

After this guide: Given a study or a graph, you should be able to name the variables, the design and the threats to its validity, judge whether it shows experimental control, and choose the design that best answers a stated question under stated constraints.

Practise area D