Meta-analysis of single-case experimental designs

A single-case experimental design (SCED) repeatedly measures one subject (or a small number of subjects) across phases — typically a baseline phase (A) without intervention and a treatment phase (B) with it — instead of comparing groups of subjects. SCEDs are common in fields like special education, clinical psychology, and behavioral intervention research, where recruiting large samples is impractical.

Why standard effect sizes don't apply

Cohen's d, odds ratios, and correlations (see Methods and formulas) are all built on between-subjects variance — they compare groups, or a group to itself before/after. SCED data has no such structure: it is one subject's repeated measurements, which are typically autocorrelated (today's value depends on yesterday's), so the standard formulas' assumptions don't hold.

Overlap-based effect sizes

Meta-analysis of SCEDs instead relies on effect sizes that quantify how much the baseline (A) and intervention (B) phase data overlap — less overlap means a clearer treatment effect:

• PND (Percentage of Non-overlapping Data): the percentage of intervention-phase data points that exceed the most extreme baseline-phase data point. Simple to compute, but sensitive to a single outlier baseline point.
• NAP (Non-overlap of All Pairs): the percentage of all possible baseline/intervention point pairs where the intervention point shows improvement — closely related to the area under a ROC curve.
• Tau-U: extends NAP by additionally correcting for any improving trend already present within the baseline phase, so it isn't mistaken for a treatment effect.

Aggregating across subjects and studies

Once an overlap index is computed for each subject (and each study may contribute several subjects, echoing the same dependency issue discussed for multilevel models), a pooled estimate can be obtained by averaging weighted by the number of data points, or through a multilevel model that accounts for subjects nested within studies.