Meta-analysis references

Books on meta-analysis

Cooper, H. (2016). Research synthesis and meta-analysis: A step-by-step approach, 5th edition. Thousand Oaks, CA: Sage.

Kline, R. B. (2013). Beyond significance testing: Statistics reform in the behavioral sciences, 2nd edition. Washington, DC: American Psychological Association.

Card, N. A. (2012). Applied meta-analysis for social science research. Guilford, New York.

Cumming, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. New York: Routledge.

Ellis, P. D. (2010). The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results. Cambridge: Cambridge University Press.

Cooper, H., Hedges, L. V., & Valentine, J. (Eds.) (2009). The handbook of research synthesis, 2nd edition. New York: Russell Sage.

Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester: John Wiley & Sons, Ltd.

Higgins, J. P. T., & Green, S. (Eds.) (2008). Cochrane handbook for systematic reviews of interventions. Chichester: John Wiley & Sons, Ltd.

Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences. Malden, MA: Blackwell Publishing.

Whitehead, A. (2003). Meta-analysis of controlled clinical trial. Chichester: John Wiley & Sons, Ltd.

Lipsey, M., & Wilson, D. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.

Sutton, A. J., Abrams, K. R., Jones, D. R., Sheldon, T. A., & Song, F. (2000). Methods for meta-analysis in medical research. Chichester: John Wiley & Sons, Ltd.

Stangl, D. K., & Berry, D. A. (2000). Meta-analysis in medicine and health policy. New York: Marcel Dekker, Inc.

Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press.

Introductory articles on meta-analysis

Crocetti, E. (2016). Systematic reviews with meta-analysis: Why, when, and how? Emerging Adulthood, 4 (1), 3-18.

Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25, 7-29.

Reed, J. G., & Baxter, P. M. (2009). Using reference databases. In H. Cooper, L. V. Hedges, & J. Valentine (Eds.), The handbook of research synthesis (pp. 73-101). New York: Russell Sage.

Gliner, J. A., Morgan, G. A., Harmon, R. J. (2003). Meta-analysis: Formulation and interpretation. Journal of American Academy of Child and Adolescent Psychiatry, 42, 1376-1379.

Bushman, B. J., & Wells, G. L. (2001). Narrative impressions of literature: The availability bias and the corrective properties of meta-analytic approaches. Personality and Social Psychology Bulletin, 27 (9), 1123-1130.

Allen, I. E., & Olkin, I. (1999). Estimating time to conduct a meta-analysis from number of citations retrieved. Journal of the American Medical Association, 282 (7), 634-635.

History of meta-analysis

Glass, G. V (2000). Meta-analysis at 25. Retrieved from https://nepc.colorado.edu/blog/meta-analysis.

Hock, R. R. (1999). Forty studies that changed psychology: Explorations into the history of psychological research. Upper Saddle River, NJ: Pearson Education.

Hunt, M. (1997). How science takes stock: The story of meta-analysis. NY: Russell Sage Foundation.

Eysenck, H. J. (1978). An exercise in mega-silliness. American Psychologist, 33, 517.

Glass, G. V (1978). Integrating findings: The meta-analysis of research. Review of Research in Education, 5, 351-379.

Smith, M. L., & Glass, G. V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist, 32, 752-760.

Glass, G. V (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3-8.

Eysenck, H. J. (1965). The effects of psychotherapy. International Journal of Psychiatry, 1, 99-178.

Eysenck, H. J. (1952). The effects of psychotherapy: An evaluation. Journal of Consulting Psychology, 16, 319-324.

Effect size calculation and conversion

Aloe, A. M., & Becker, B. J. (2012). An effect size for regression predictors in meta-analysis. Journal of Educational and Behavioral Statistics, 37, 278-297.

Becker, B. J., & Wu, M. (2007). The synthesis of regression slopes in meta-analysis. Statistical Science, 22, 414-429.

Tierney, J. F., Stewart, L. A., Ghersi, D., Burdett, S., & Sydes, M. R. (2007). Practical methods for incorporating summary time-to-event data into meta-analysis. Trials, 8, 1-16.

Peterson, R. A., & Brown, S. P. (2005). On the use of beta coefficients in meta-analysis. Journal of Applied Psychology, 90, 175-181.

Williamson, P. R., Smith, C. T., Hutton, J. L., & Marson, A. G. (2002). Aggregate data meta-analysis with time-to-event outcomes. Statistics in Medicine, 21, 3337-3351.

Chinn, S. (2000). A simple method for converting an odds ratio to effect size for use in meta-analysis. Statistics in Medicine, 19 (22), 3127-3131.

Parmar, M. K., Torri, V., & Stewart, L. (1998). Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints. Statistics in Medicine, 17, 2815-2834.

Rupinski, M. T., & Dunlap, W. P. (1996). Approximating Pearson product-moment correlations from Kendall's tau and Spearman's rho. Educational and Psychological Measurement, 56, 419-429.

Fleiss, J. L. (1994). Measures of effect size for categorical data. In H. Cooper, L. V. Hedges, & J. Valentine (Eds.), The handbook of research synthesis (pp. 245-260). New York: Russell Sage.

Kendall, M. G., & Gibbons, J. D. (1990). Rank correlation methods, 5th edition. Oxford University Press.

Fixed vs. random effects models

Hedges, L. V., & Vevea, J. L. (1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3, 486-504.

DerSimonian, R., & Laird, N. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7 (3), 177-188.

Hedges, L. V. (1983). A random effects model for effect sizes. Psychological Bulletin, 93 (2), 388-395.

Heterogeneity in meta-analysis

Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009). A re-evaluation of random-effects meta-analysis. Journal of the Royal Statistical Society: Series A, 172 (1), 137-159.

Viechtbauer, W. (2007). Accounting for heterogeneity via random-effects models and moderator analyses in meta-analysis. Journal of Psychology, 215 (2), 104-121.

Huedo-Medina, T. B., Sánchez-Meca, J., Marin-Martinez, F., & Botella, J. (2006). Assessing heterogeneity in meta-analysis: Q Statistic or I² index? Psychological Methods, 11 (2), 193-206.

Sidik, K., & Jonkman, J. N. (2005). Simple heterogeneity variance estimation for meta-analysis. Journal of the Royal Statistical Society: Series C, 54 (2), 367-384.

Viechtbauer, W. (2005). Bias and efficiency of meta-analytic variance estimators in the random-effects model. Journal of Educational and Behavioral Statistics, 30 (3), 261-293.

Higgins, J., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327, 557-560.

Higgins, J., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21, 1539-1558.

Morris, C. N. (1983). Parametric empirical Bayes inference: Theory and applications. Journal of the American Statistical Association, 78 (381), 47-55.

Paule, R. C., & Mandel, J. (1982). Consensus values and weighting factors. Journal of Research of the National Bureau of Standards, 87 (5), 377-385.

Harville, D. A. (1977). Maximum likelihood approaches to variance component estimation and to related problems. Journal of the American Statistical Association, 72 (358), 320-338.

Cochran, W. G. (1954). The combination of estimates from different experiments. Biometrics, 10 (1), 101-129.

Moderators, meta-regression, and advanced models

Pustejovsky, J. E., & Tipton, E. (2018). Small-sample methods for cluster-robust variance estimation and hypothesis testing in fixed effects models. Journal of Business & Economic Statistics, 36 (4), 672-683.

Stanley, T. D., & Doucouliagos, H. (2017). Neither fixed nor random: Weighted least squares meta-regression. Research Synthesis Methods, 8 (1), 19-42.

Tipton, E. (2015). Small sample adjustments for robust variance estimation with meta-regression. Psychological Methods, 20 (3), 375-393.

Stanley, T. D., & Doucouliagos, H. (2014). Meta-regression approximations to reduce publication selection bias. Research Synthesis Methods, 5 (1), 60-78.

Van den Noortgate, W., López-López, J. A., Marín-Martínez, F., & Sánchez-Meca, J. (2013). Three-level meta-analysis of dependent effect sizes. Behavior Research Methods, 45 (2), 576-594.

Knapp, G., & Hartung, J. (2003). Improved tests for a random effects meta-regression with a single covariate. Statistics in Medicine, 22 (17), 2693-2710.

Publication bias in meta-analysis

Rothstein, H. R., & Hopewell, S. (2009). Grey literature. In H. Cooper, L. V. Hedges, & J. Valentine (Eds.), The handbook of research synthesis (pp. 103-125). New York: Russell Sage.

Tetzlaff, J., Moher, D., Pham, B., & Altman, D. (2006). Survey of views on including grey literature in systematic reviews. 14th Cochrane Colloquium, Dublin, Ireland.

Dickersin, K. (2005). Publication bias: Recognizing the problem, understanding its origins and scope, and preventing harm. In H. R. Rothstein, A. J. Sutton, M. Borenstein (Eds.), Publication bias in meta-analysis (pp. 127-144). Chichester: John Wiley & Sons, Ltd.

Duval, S. (2005). The trim and fill method. In H. R. Rothstein, A. J. Sutton, M. Borenstein (Eds.), Publication bias in meta-analysis (pp. 11-33). Chichester: John Wiley & Sons, Ltd.

Halpern, S. D., & Berlin, J. A. (2005). Beyond conventional publication bias: Other determinants of data suppression. In H. R. Rothstein, A. J. Sutton, M. Borenstein (Eds.), Publication bias in meta-analysis (pp. 303-317). Chichester: John Wiley & Sons, Ltd.

Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.) (2005). Publication bias in meta-analysis. Chichester: John Wiley & Sons, Ltd.

Sterne, J. A., Becker, B. J., & Egger, M. (2005). The funnel plot. In H. R. Rothstein, A. J. Sutton, M. Borenstein (Eds.), Publication bias in meta-analysis (pp. 75-98). Chichester: John Wiley & Sons, Ltd.

Rosenberg, M. S. (2005). The file-drawer problem revisited: A general weighted method for calculating fail-safe numbers in meta-analysis. Evolution, 59 (2), 464-468.

Sterne, J. A. C., & Egger, M. (2005). Regression methods to detect publication and other bias in meta-analysis. In H. R. Rothstein, A. J. Sutton, M. Borenstein (Eds.), Publication bias in meta-analysis (pp. 99-110). Chichester: John Wiley & Sons, Ltd.

Becker, B. J. (2005). Failsafe N or file-drawer number. In H. R. Rothstein, A. J. Sutton, M. Borenstein (Eds.), Publication bias in meta-analysis (pp. 111-125). Chichester: John Wiley & Sons, Ltd.

Krzyzanowska, M. K., Pintilie, M., & Tannock, I. F. (2003). Factors associated with failure to publish large randomized trials presented at an oncology meeting. Journal of the American Medical Association, 290, 495-501.

Sterne, J. A., & Egger, M. (2001). Funnel plots for detecting bias in meta-analysis: Guidelines on choice of axis. Journal of Clinical Epidemiology, 54, 1046-1055.

Duval, S., & Tweedie, R. (2000). A nonparametric 'trim and fill' method of accounting for publication bias in meta-analysis. Journal of American Statistical Association, 95, 89-98.

Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56 (2), 455-463.

Cooper, H., DeNeve, K., & Charlton, K. (1997). Finding the missing science: The fate of studies submitted for review by a human subjects committee. Psychological Methods, 2, 447-452.

Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315, 629-634.

Berlin, J. A. (1997). Does blinding of readers affect the results of meta-analyses? The Lancet, 350, 185-186.

Gleser, L. J., & Olkin, I. (1996). Models for estimating the number of unpublished studies. Statistics in Medicine, 15, 2493-2507.

Vevea, J. L., & Hedges, L. V. (1995). A general linear model for estimating effect size in the presence of publication bias. Psychometrika, 60 (3), 419-435.

Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50, 1088-1101.

Dickersin, K., Chan, S., Chalmers, T. C., Sacks, H. S., & Smith Jr., H. (1987). Publication bias and clinical trials. Controlled Clinical Trials, 8, 343-353.

Light, R. J., & Pillemer, D. B. (1984). Summing up: The science of reviewing research. Cambridge, MA: Harvard University Press.

Orwin, R. G. (1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8, 157-159.

Smith, M. L. (1980). Publication bias and meta-analysis. Evaluation in Education, 4, 22-24.

Rosenthal, R. (1979). The "file drawer problem" and tolerance for null results. Psychological Bulletin, 86, 638-641.

Mahoney, M. J. (1977). Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy & Research, 1, 161-175.

Greenwald, A. G. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin, 82, 1-20.

Power of statistical tests in meta-analysis

Hedges, L. V., & Pigott, T. D. (2004). The power of statistical tests for moderators in meta-analysis. Psychological Methods, 9, 426-445.

Hedges, L. V., & Pigott, T. D. (2001). The power of statistical tests in meta-analysis. Psychological Methods, 6, 203-217.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences. New York, NY: Academic Press.

The meta-analytic approach of Hunter and Schmidt

Hunter, J. E., & Schmidt, F. L. (2014). Methods of meta-analysis: Correcting error and bias in research findings, 3rd edition. Newbury Park, CA: Sage Publications.

Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage Publications.

R packages for meta-analysis

Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36 (3), 1-48.

Study quality and risk-of-bias assessment

Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., ... Higgins, J. P. T. (2019). RoB 2: A revised tool for assessing risk of bias in randomised trials. BMJ, 366, l4898.

Publication guidelines for systematic reviews and meta-analyses

American Psychological Association (2010). Publication manual of the American Psychological Association, 6th edition. Washington, DC: American Psychological Association.

Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., et al. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. PLoS Med, 6 (7), e1000100.

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., The PRISMA Group (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med, 6 (6), e1000097.

Stroup, D. F., Berlin, J. A., Morton, S. C., Olkin, I., Williamson, G. D., Rennie, D., ... Thacker, S. B. (2000). Meta-analysis of observational studies in epidemiology: A proposal for reporting. Journal of the American Medical Association, 283 (15), 2008-2012.