Alternative measures of between-study heterogeneity in meta-analysis: Reducing the impact of outlying studies

Biometrics. 2017 Mar;73(1):156-166. doi: 10.1111/biom.12543. Epub 2016 May 11.


Meta-analysis has become a widely used tool to combine results from independent studies. The collected studies are homogeneous if they share a common underlying true effect size; otherwise, they are heterogeneous. A fixed-effect model is customarily used when the studies are deemed homogeneous, while a random-effects model is used for heterogeneous studies. Assessing heterogeneity in meta-analysis is critical for model selection and decision making. Ideally, if heterogeneity is present, it should permeate the entire collection of studies, instead of being limited to a small number of outlying studies. Outliers can have great impact on conventional measures of heterogeneity and the conclusions of a meta-analysis. However, no widely accepted guidelines exist for handling outliers. This article proposes several new heterogeneity measures. In the presence of outliers, the proposed measures are less affected than the conventional ones. The performance of the proposed and conventional heterogeneity measures are compared theoretically, by studying their asymptotic properties, and empirically, using simulations and case studies.

Keywords: Absolute deviation; Heterogeneity; I2 statistic; Meta-analysis; Outliers.

Publication types

  • Research Support, N.I.H., Extramural

MeSH terms

  • Aged
  • Analysis of Variance*
  • Computer Simulation
  • Data Interpretation, Statistical*
  • Exercise
  • Female
  • Hip Fractures / mortality
  • Humans
  • Intra-Abdominal Fat
  • Male
  • Meta-Analysis as Topic*
  • Statistics as Topic