Misleading Funnel Plot for Detection of Bias in Meta-Analysis

J Clin Epidemiol. 2000 May;53(5):477-84. doi: 10.1016/s0895-4356(99)00204-8.

Abstract

Publication and other forms of selection biases pose a threat to the validity of meta-analysis. Funnel plots are usually used to detect such biases; asymmetrical plots are interpreted to suggest that biases are present. Using 198 published meta-analyses, we demonstrate that the shape of a funnel plot is largely determined by the arbitrary choice of the method to construct the plot. When a different definition of precision and/or effect measure were used, the conclusion about the shape of the plot was altered in 37 (86%) of the 43 meta-analyses with an asymmetrical plot suggesting selection bias. In the absence of a consensus on how the plot should be constructed, asymmetrical funnel plots should be interpreted cautiously. These findings also suggest that the discrepancies between large trials and corresponding meta-analyses and heterogeneity in meta-analyses may also be determined by how they are evaluated.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Data Interpretation, Statistical
  • Effect Modifier, Epidemiologic
  • Humans
  • Meta-Analysis as Topic*
  • Reproducibility of Results
  • Risk
  • Selection Bias*
  • Statistics as Topic / methods*