Statistical models: fixed-effect, random-effects, and multilevel

Once every study has its own effect size (see Methods and formulas), combining them into a single pooled estimate requires a statistical model. The three models below differ only in which sources of variance they account for when weighting each study.

In all formulas below, k is the number of studies, and each study i contributes an effect size (Y with subscript i) and its variance (v with subscript i).

Fixed-effect model

The fixed-effect model assumes there is one true effect size shared by every study — any difference between studies is treated as sampling error. Every effect size is weighted by the inverse of its variance, so more precise studies (larger samples, smaller variance) count more. This inverse-variance weighting scheme for combining estimates across studies goes back to Cochran (1954) — see the reference list for further reading:

w_{i} = \frac{1}{v_{i}}

The pooled effect is the weighted mean, and its own variance is the inverse of the summed weights:

M = \frac{\sum_{i = 1}^{k} w_{i} Y_{i}}{\sum_{i = 1}^{k} w_{i}}

V_{M} = \frac{1}{\sum_{i = 1}^{k} w_{i}}

Random-effects model

When the assumption of one shared true effect doesn't hold — studies differ enough in population, intervention, or setting that each is estimating its own true effect — the random-effects model adds a between-studies variance component, τ². It is estimated here with the method introduced by DerSimonian & Laird (1986), the most widely used way to fit this model — see the reference list for further reading:

τ^{2} = \frac{Q - (k - 1)}{C}, C = \sum_{i = 1}^{k} w_{i} - \frac{\sum_{i = 1}^{k} w_{i}^{2}}{\sum_{i = 1}^{k} w_{i}}

Each study's weight then also accounts for τ², shrinking the influence of very large studies relative to the fixed-effect model:

w_{i}^{*} = \frac{1}{v_{i} + τ^{2}}

Multilevel (three-level) model

Both models above assume one independent effect size per study. That assumption breaks when a study reports several effect sizes that share the same sample — several outcomes, several timepoints, or several subscales — because those effect sizes are statistically dependent, not independent draws. Ignoring the dependency understates the true variance of the pooled estimate: it silently double-counts information from the same participants, so confidence intervals come out too narrow and significance tests too liberal.

A multilevel (three-level) model, as formalized for meta-analysis by Van den Noortgate et al. (2013), adds a variance component for this extra layer of nesting:

Level 1 — sampling variance of each individual effect size, as in the models above.
Level 2 — variance between effect sizes within the same study (how much a study's own outcomes/timepoints vary around that study's true effect).
Level 3 — variance between studies (how much studies vary around the overall true effect).

{Var}_{(Y_{i j})} = v_{i j} + σ_{(2)}^{2} + σ_{(3)}^{2}

Level 2 and level 3 together generalize the single τ² of the random-effects model into two components — one for within-study heterogeneity, one for between-study heterogeneity.

Three-level meta-analysis: level 3 variance between studies, level 2 variance between effect sizes nested within each study, level 1 sampling variance of each effect size

An alternative to fitting the full three-level model is robust variance estimation (RVE): fit a simpler working model (e.g., ordinary random-effects, or an assumed within-study correlation) and correct the standard errors afterward to account for the dependency, without needing to estimate the level-2/level-3 split explicitly. RVE is a common choice when the number of studies is small, since three-level models need enough studies to estimate all variance components reliably.

A widely used middle ground, sometimes called the correlated-effects working model, doesn't try to estimate the level-2 variance at all: it assumes a plausible within-study correlation (e.g., 0.5 — a common default when the true value isn't known) instead of estimating it, and only estimates the between-study variance τ². This is a reasonable simplification for two reasons. First, with the modest number of studies typical of most meta-analyses, there usually isn't enough information to reliably estimate two separate variance components at once — the extra parameter the full model asks for often isn't well identified by the data anyway. Second, this working model is normally paired with robust variance estimation for the final standard errors, which corrects for the model's covariance structure being wrong (including the assumed correlation) — so the assumption only has to be plausible, not exactly right, for the reported inference to stay valid. ProMeta implements this approach: a correlated-effects working model paired with cluster-robust (CR2) variance estimation.

Choosing between them

Use the fixed-effect model only when there is a substantive reason to believe every study estimates the same true effect (e.g., tightly controlled replications) — in practice, this is rare.
Default to the random-effects model whenever studies differ in ways that could plausibly shift the true effect — this is the common choice in most fields.
Reach for the multilevel model (or RVE) whenever any study contributes more than one effect size to the same analysis, on top of whichever variance structure (fixed or random) applies at the study level.

See Heterogeneity for how to test whether the fixed-effect assumption actually holds, before deciding between the first two.