Study guide · Area C

Measurement, Data Display, and Interpretation

21 of the exam's 175 scored questions (12%) · 12 tasks

This area covers how behavior is defined, measured, checked, graphed and read: operational definitions, measures of occurrence and time, continuous and discontinuous procedures, efficiency, validity and reliability, choosing a measure, graphing and visual analysis, and measuring procedural integrity. Questions usually give a behavior and a setting and ask for the best measure, the error a procedure will introduce, the right IOA formula, or what a graph shows.

A study feature written by ExamDragon, not part of the real exam. Each heading is one of the BACB's tasks for this content area (12 of them), in the BACB's words.

C.1Create operational definitions of behavior.

An operational definition describes a target behavior in observable, measurable terms so that two observers would agree on every occurrence; it says what the person does, not what they are or feel.
A good definition is objective (refers only to observable features), clear (readable and unambiguous) and complete (states what is included and what is excluded, with examples and nonexamples of borderline cases).
Function-based definitions designate responses by their effect on the environment (any response that results in the toy being removed from a peer's hands); topography-based definitions designate responses by their form (pressing palm to another person's body with force).
Function-based definitions are generally preferred when the function matters, because they include all forms that serve it; topography-based definitions are used when the form itself matters or the function cannot be observed.
The definition should also state onset and offset criteria (when an occurrence begins and ends), which are needed for duration, latency and interval data.
Labels such as aggressive, frustrated, off task or tantrum are not definitions; they must be replaced by the observable responses they refer to.

In practice: Instead of defining the target as tantrums, a behavior analyst writes: any instance of screaming above conversational volume, dropping to the floor or kicking furniture, lasting 3 seconds or more; an occurrence ends after 10 seconds with none of these. Crying without screaming is excluded. Two staff now record the same episodes.

Where questions turn: A definition that names an inner state or a label (angry, defiant, upset) is not operational, however detailed the rest of it is.

Practise C.1

C.2Distinguish among direct, indirect, and product measures of behavior.

Direct measurement records the behavior itself, at the time and place it occurs, as the target behavior (observing and counting hand raises in class).
Indirect measurement records something other than the behavior of interest or relies on reports at a later time: interviews, rating scales, checklists or self-reports of behavior that happened elsewhere.
Indirect measures require inferences about the relation between what was recorded and the actual behavior, so they are less accurate and are used when direct measurement is not possible.
Measurement by permanent product records an effect the behavior leaves on the environment after it has occurred: completed worksheets, assembled units, clean dishes, wrappers in a bin.
Permanent product measurement is useful when the behavior is hard to observe or occurs when no observer is present, but it requires that each product is produced by the target behavior and the target person, and it says little about how the product was produced.

In practice: To evaluate a student's math work, a teacher counts the correct answers on the completed worksheet (permanent product). To measure his hand flapping, she observes and counts it during class (direct). To learn about flapping at home, she uses a parent rating scale (indirect).

Where questions turn: A permanent product is still a measure of the behavior's real effect, recorded after the fact; a report or rating of behavior is indirect, even if completed by someone who saw it.

Practise C.2

C.3Measure occurrence.

Count is the simple tally of the number of responses observed; it is meaningful only when the observation time is held constant or reported with it.
Rate (frequency, in the current usage of Cooper, Heron and Heward) is count per unit of time (responses per minute or per hour), which allows comparison across observations of different lengths.
When responses have an opportunity requirement, express rate as correct and incorrect per minute or report the number of opportunities; for discrete trials, percentage correct or trials to criterion are typical.
Percentage is a ratio of responses to opportunities multiplied by 100; it shows proportion but hides the number of opportunities and the rate, and it has a ceiling of 100 percent.
Celeration is the change in rate over time (count per unit of time per unit of time), plotted on a Standard Celeration Chart; it shows acceleration or deceleration of performance.
Count and rate are best for behaviors that have clear beginnings and ends and occur for roughly constant durations; for behaviors that vary greatly in length, temporal measures may be needed.

In practice: A student hits peers 12 times in a 60-minute session on Monday and 8 times in a 20-minute session on Tuesday. Counts alone suggest improvement, but rates (0.2 per minute versus 0.4 per minute) show the behavior doubled.

Where questions turn: Comparing counts from observations of different lengths is invalid; convert to rate when observation periods differ.

Practise C.3

C.4Measure temporal dimensions of behavior (e.g., duration, latency, interresponse time).

Duration is the amount of time from the onset to the offset of a response; it can be reported as total duration per session or as duration per occurrence.
Response latency is the time from the onset of a stimulus (such as an instruction) to the initiation of the response.
Interresponse time (IRT) is the time elapsing between two successive responses, measured from the end of one response to the beginning of the next.
Duration suits behaviors where length matters (time on task, tantrum length, time spent in social play); latency suits behaviors where speed of starting matters (beginning work after a request, compliance); IRT suits behaviors whose pace matters (bites of food, cigarettes smoked).
IRT and rate are inversely related: as the time between responses shortens, rate increases.
Each temporal measure requires precise onset and offset criteria in the operational definition.

In practice: A teacher measures the time from the instruction take out your book to the moment a student touches the book (latency), how long he reads once he begins (duration), and, for a student who eats too fast, the seconds between one bite and the next (IRT) to target with a pacing program.

Where questions turn: Latency starts at the antecedent stimulus and ends at the start of the response; IRT starts at the end of one response and ends at the start of the next.

Practise C.4

C.5Distinguish between continuous and discontinuous measurement procedures.

Continuous measurement detects every instance of the behavior during the observation period: event recording (count, rate), duration, latency and IRT.
Discontinuous measurement does not record every instance; it samples behavior in intervals or at moments: whole-interval recording, partial-interval recording and momentary time sampling.
Continuous measurement gives the most accurate account of what occurred, but it requires observers who can attend throughout and behaviors with clear onsets and offsets.
Discontinuous methods produce estimates, usually expressed as the percentage of intervals in which the behavior was scored, not true counts or durations.
Discontinuous methods are chosen when continuous recording is impractical: high-rate behavior, behavior without clear boundaries, several people observed at once, or an observer with other duties.

In practice: A behavior analyst counts every instance of a child's spitting during a 30-minute observation (continuous event recording). For the same child's humming, which starts and stops unpredictably and often blends together, she uses 10-second partial-interval recording (discontinuous).

Where questions turn: Interval and time sampling data are estimates expressed as a percentage of intervals; they are not counts of the behavior and cannot be converted back into true rate or duration.

Practise C.5

C.6Design and apply discontinuous measurement procedures (e.g., interval recording, time sampling).

Whole-interval recording: the observation is divided into equal intervals and the behavior is scored only if it occurs throughout the entire interval; it tends to underestimate occurrence and suits behaviors you want to increase that should be continuous, such as on-task behavior.
Partial-interval recording: the behavior is scored if it occurs at any time during the interval; it tends to overestimate total duration and suits behaviors you want to decrease, since a low score reflects real improvement.
Momentary time sampling: the behavior is scored only if it is occurring at the moment an interval ends; it can over- or underestimate, misses brief behaviors, and is practical for continuous behaviors or for observing several people (planned activity check, PLACHECK, applies it to a group).
Shorter intervals generally make estimates more accurate; the interval should be shorter than the typical duration and interresponse time of the behavior.
To design a procedure: define the behavior, choose interval length, decide the scoring rule (whole, partial, momentary), use a timing cue (vibrating timer or audio prompt) so observers know when intervals end, and report data as percentage of intervals.
Some designs alternate observe intervals with record intervals (such as 10 seconds observe, 5 seconds record) to give the observer time to write.

In practice: A teacher wants to know how much of a reading period a student spends engaged. She divides 20 minutes into 30-second intervals and scores an interval only if he reads for all of it (whole interval), knowing this will understate engagement, and accepts that bias because her goal is to increase reading.

Where questions turn: Partial-interval recording overestimates and whole-interval recording underestimates; choose the method whose bias works against showing false improvement for the behavior's direction of change.

Practise C.6

C.7Measure efficiency (e.g., trials to criterion, cost-benefit analysis, training duration).

Efficiency measures describe how quickly and at what cost a skill is learned or an intervention produces results, beyond whether it works at all.
Trials to criterion is the number of response opportunities needed to reach a mastery criterion; it is used to compare instructional methods or to track learning speed across sets of targets.
Training duration (or sessions or minutes to criterion) measures the total instructional time required; one method may need fewer trials but more time per trial, so both can matter.
Cost-benefit analysis weighs resources (staff time, materials, training, money) against the outcomes produced, to select among interventions with similar effects.
Efficiency comparisons should be made under comparable conditions (same learner, similar targets, same criterion) so the difference can be attributed to the method.

In practice: A clinic compares two prompting methods for teaching sight words to the same learner, each with a matched word set. Method A reaches 90 percent correct across two sessions in 60 trials and 25 minutes; method B takes 90 trials and 30 minutes. The team chooses method A as the more efficient.

Where questions turn: An intervention can be effective without being efficient; efficiency is about the trials, time or resources needed to reach the same outcome.

Practise C.7

C.8Evaluate the validity and reliability of measurement procedures.

Validity: the data measure what they are intended to measure, a socially significant behavior and the dimension of it relevant to the question. Threats include measuring a proxy instead of the behavior, measuring the wrong dimension, and measuring at the wrong times or places.
Accuracy: the observed values match the true values of the behavior as it actually occurred. It is assessed by comparing data with a true value, such as a video record scored very carefully.
Reliability: the measurement yields the same values when the same behavior is measured repeatedly; reliable data can still be inaccurate if the error is consistent.
Interobserver agreement (IOA) is the degree to which two independent observers report the same values; it indicates believability and helps detect observer drift and poorly written definitions, but agreement does not prove accuracy.
Common IOA methods: total count, mean count-per-interval, exact count-per-interval and trial-by-trial for event data; total duration and mean duration-per-occurrence for time data; interval-by-interval, scored-interval and unscored-interval for interval data. By convention (a guideline, not a rule), at least 80 percent agreement is generally acceptable and 90 percent or more is preferred, with IOA gathered in at least 20 percent of sessions (25 to 33 percent preferred) across all conditions.
Sources of measurement error include poorly designed systems, inadequate observer training, observer drift (definitions applied differently over time), reactivity (the person behaves differently because they are observed) and observer expectations.

In practice: Two observers record a student's call-outs: one counts 18 and the other 20. Total count IOA is 18 divided by 20, multiplied by 100, or 90 percent. The analyst notes that the two could agree perfectly and both still miss whispered call-outs, so she also checks a sample of video against the data for accuracy.

Where questions turn: High IOA shows that two observers agree, not that either recorded the behavior accurately, and total count IOA can be high even when the observers scored different occurrences.

Practise C.8

C.9Select a measurement procedure to obtain representative data that accounts for the critical dimension of the behavior and environmental constraints.

Start with the question: which dimension of the behavior is socially significant (how often, how long, how quickly, how much, how well, where)?
Match the measure to that dimension: rate for repeatable discrete responses, duration for behaviors whose length matters, latency for speed of starting, IRT for pacing, percentage or trials to criterion for responses to defined opportunities, magnitude or topography where force or form matters.
Then account for constraints: observer availability, number of people observed, the visibility of the behavior, whether it leaves a product, its rate and whether it has clear boundaries.
Use continuous measures when feasible; choose discontinuous methods knowingly, pick the one whose bias is acceptable, and shorten intervals for accuracy.
Make the data representative: schedule observations at the times and places where the behavior matters and samples that are long and frequent enough to show its real level.
Avoid artifacts: a measure with a ceiling or floor, an observation period too short to detect change, or a method insensitive to the change you expect.

In practice: A teacher wants to reduce how long a student is out of his seat; outbursts are rare but long. Counting out-of-seat episodes would hide a reduction in length, so she measures total duration with a stopwatch during the two periods when it happens most, and adds a duration-per-occurrence summary.

Where questions turn: The best measure is the one that captures the dimension of concern under the real constraints, not the most precise or most common measure.

Practise C.9

C.10Graph data to communicate relevant quantitative relations (e.g., equal-interval graphs, bar graphs, cumulative records).

The equal-interval line graph is the most common display in applied work: equal distances on each axis represent equal amounts, the horizontal axis shows time or sessions, the vertical axis the behavior, and phase change lines separate conditions with data paths not connected across them.
A bar graph compares discrete sets of data or summarizes performance across conditions or categories (mean rate by condition); it does not show variability or trend within a condition.
A cumulative record shows the total number of responses accumulated over time; the slope shows the rate (steeper is faster), a flat line shows no responding, and the line can never go down.
A ratio chart, such as the Standard Celeration Chart, uses a logarithmic vertical axis so equal distances show equal proportional changes; it displays celeration and allows comparisons across very different rates.
A scatterplot shows the relative distribution of individual measures and is used, among other things, to show when during the day a behavior occurs.
Good graphs include clearly labeled axes with units, condition labels, phase lines, data points that are visible and a scale that does not distort the change.

In practice: A behavior analyst graphs a learner's daily rate of manding during baseline and intervention on a line graph with a phase change line, so the team can see level and trend. For the funding report, she adds a bar graph of the mean rate per phase, and for the supervisor a cumulative record of total mands per week.

Where questions turn: On a cumulative record, a flat segment means no responding, not steady responding; rate is shown by the slope, not the height.

Practise C.10

C.11Interpret graphed data.

Visual analysis looks at three properties within each condition: level (the value on the vertical axis around which data converge, often a mean or median), trend (the overall direction and steepness of the data path) and variability (how much the data points differ from one another).
Between conditions, look at the immediacy of change at the phase change, the overlap of data points across conditions, and the consistency of patterns across similar phases.
A stable baseline is needed before introducing a change; a baseline trending in the direction of the expected treatment effect makes any later improvement hard to attribute to the intervention.
Highly variable data call for more data points or a search for the variables causing the variability before a conclusion is drawn.
Visual analysis judges whether the change is large, consistent and replicated enough to show experimental control and to be meaningful, not whether it would pass a statistical test.
Read the axes first: scale, units and break lines change the picture, and a tight or stretched vertical axis can exaggerate or hide effects.

In practice: A graph shows baseline rates of self-injury between 8 and 10 per hour with no trend, then an immediate drop to between 1 and 2 per hour in intervention with no overlapping points. The analyst describes a large immediate change in level, low variability and no overlap, strong evidence of a change between conditions.

Where questions turn: A baseline that is already moving in the therapeutic direction makes an intervention effect uninterpretable, even if the intervention phase looks good.

Practise C.11

C.12Select a measurement procedure to obtain representative procedural integrity data that accounts for relevant dimensions (e.g., accuracy, dosage) and environmental constraints.

Procedural (treatment) integrity is the extent to which the independent variable is implemented as planned; without it, outcomes cannot be attributed to the intervention.
Dimensions to measure include accuracy (each component done correctly), adherence (steps done in order and on schedule), dosage or exposure (how often and how long the intervention was delivered) and errors of omission (a step left out) and commission (a step added or done wrongly).
Methods: direct observation with a component checklist, video review, permanent products (completed token boards, data sheets) and, least accurate, self-report by implementers.
Integrity can be summarized as the percentage of components done correctly per session or the percentage of opportunities on which a specific component was correct; the second shows which component fails.
Choose the method by constraints: the observer's time, how often the intervention occurs, whether the components leave products, and how intrusive observation is; sample enough sessions across implementers and settings to be representative.
Plan integrity measurement from the start, with clear operational definitions of each component, and collect it throughout, not only when outcomes are poor.

In practice: Parents implement a functional communication program at home. Because the analyst can visit only weekly, she combines a weekly live checklist of the 6 components with a review of two short videos the parents record each week, scoring the percentage of opportunities on which they honored a request within 5 seconds.

Where questions turn: Integrity data measure the implementer's behavior, not the client's; good client outcomes do not show the procedure was implemented as written.

Practise C.12

Key terms

Operational definition: A description of behavior in observable, measurable terms that observers can agree on.
Rate: Count of responses per unit of observation time.
Celeration: The change in rate of responding over time.
Latency: Time from the onset of a stimulus to the start of the response.
Interresponse time: Time between the end of one response and the start of the next.
Whole-interval recording: Scoring an interval only if the behavior occurred throughout it; tends to underestimate.
Partial-interval recording: Scoring an interval if the behavior occurred at any time in it; tends to overestimate.
Momentary time sampling: Scoring whether the behavior is occurring at the moment each interval ends.
Permanent product: A lasting effect of behavior on the environment, measured after the behavior has occurred.
Interobserver agreement: The degree to which independent observers report the same measured values.
Observer drift: Unintended change over time in how an observer applies a definition.
Reactivity: Change in behavior caused by the person knowing they are being observed.
Cumulative record: A graph of responses summed over time, in which slope shows rate.
Procedural integrity: The extent to which an intervention is implemented as designed.

After this guide: Given a behavior, a setting and a question, you should be able to write its operational definition, choose and design the measure that captures its critical dimension, compute and judge IOA and integrity data, and read level, trend and variability on a graph.

Practise area C