In meta-analyses of proportion studies, funnel plots were found to be an inaccurate method of assessing publication bias

J Clin Epidemiol. 2014 Aug;67(8):897-903. doi: 10.1016/j.jclinepi.2014.03.003. Epub 2014 Apr 29.


Objective: To assess the utility of funnel plots in assessing publication bias (PB) in meta-analyses of proportion studies.

Study design and setting: Meta-analysis simulation study and meta-analysis of published literature reporting peri-operative mortality after abdominal aortic aneurysm (AAA) repair. Data for the simulation study were stochastically generated. A literature search of Medline and Embase was performed to identify studies for inclusion in the published literature meta-analyses.

Results: The simulation study demonstrated that conventionally constructed funnel plots (log odds vs. 1/standard error [1/SE]) for extreme proportional outcomes were asymmetric despite no PB. Alternative funnel plots constructed using study size rather than 1/SE showed no asymmetry for extreme proportional outcomes. When used in meta-analyses of the mortality of AAA repair, these alternative funnel plots highlighted the possibility for conventional funnel plots to demonstrate asymmetry when there was no evidence of PB.

Conclusion: Conventional funnel plots used to assess for potential PB in meta-analyses are inaccurate for meta-analyses of proportion studies with low proportion outcomes. Funnel plots of study size against log odds may be a more accurate way of assessing for PB in these studies.

Keywords: Abdominal aortic aneurysm; Funnel plot; Meta-analysis; Peri-operative mortality; Proportional outcomes; Publication bias.

Publication types

  • Meta-Analysis

MeSH terms

  • Aortic Aneurysm, Abdominal / mortality*
  • Aortic Aneurysm, Abdominal / surgery*
  • Cardiovascular Surgical Procedures / mortality
  • Cardiovascular Surgical Procedures / statistics & numerical data
  • Data Interpretation, Statistical*
  • Humans
  • Models, Statistical*
  • Publication Bias*
  • Stochastic Processes