Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
, 58 (9), 894-901

In an Empirical Evaluation of the Funnel Plot, Researchers Could Not Visually Identify Publication Bias

Affiliations

In an Empirical Evaluation of the Funnel Plot, Researchers Could Not Visually Identify Publication Bias

Norma Terrin et al. J Clin Epidemiol.

Abstract

Background and objective: Publication bias and related biases can lead to overly optimistic conclusions in systematic reviews. The funnel plot, which is frequently used to detect such biases, has not yet been subjected to empirical evaluation as a visual tool. We sought to determine whether researchers can correctly identify publication bias from visual inspection of funnel plots in typical-size systematic reviews.

Methods: A questionnaire with funnel plots containing 10 studies each (the median number in medical meta-analyses) was completed by 41 medical researchers, including clinical research fellows in a meta-analysis class, faculty in clinical care research, and experienced systematic reviewers.

Results: On average, participants correctly identified 52.5% (95% CI 50.6-54.4%) of the plots as being affected or unaffected by publication bias. The weighted mean percent correct, which adjusted for the fact that asymmetric plots are more likely to occur in the presence of publication bias, was also low (48.3 to 62.8%, depending on the presence or absence of publication bias and heterogeneous study effects).

Conclusion: Researchers who assess for publication bias using the funnel plot may be misled by its shape. Authors and readers of systematic reviews need to be aware of the limitations of the funnel plot.

Similar articles

See all similar articles

Cited by 87 articles

See all "Cited by" articles

Publication types

LinkOut - more resources

Feedback