Invited commentary: a critical look at some popular meta-analytic methods

Am J Epidemiol. 1994 Aug 1;140(3):290-6. doi: 10.1093/oxfordjournals.aje.a117248.


Meta-analysis is essential for obtaining reproducible summaries of study results and valuable for discovering patterns among study results. A good meta-analysis will highlight and delineate the subjective components of these processes and vigorously search for sources of heterogeneity. Unfortunately, these objective are not always met by common techniques. For example, a scatterplot is an objective summarization if the data are uncensored, but inferred patterns should be regarded as subjective recognitions of the analyst, not objective data properties. Random-effects summaries encourage averaging over important data patterns, divert attention from key sources of heterogeneity, and can amplify distortions produced by publication bias; such summaries should only be used when important heterogeneity remains after a thorough search for the sources of such heterogeneity. Quality scoring adds the analyst's subjective bias to the results, wastes information, and can prevent the recognition of key sources of heterogeneity; it should be completely replaced by meta-regression on quality items (the score components).

MeSH terms

  • Bias
  • Data Interpretation, Statistical*
  • Epidemiologic Methods
  • Humans
  • Meta-Analysis as Topic*
  • Models, Statistical*
  • Quality Control
  • Regression Analysis