Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2013 Jul 5;8(7):e66844.
doi: 10.1371/journal.pone.0066844. Print 2013.

Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias - An Updated Review

Collaborators, Affiliations
Free PMC article
Review

Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias - An Updated Review

Kerry Dwan et al. PLoS One. .
Free PMC article

Abstract

Background: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias and outcome reporting bias have been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making.

Methodology/principal findings: In this update, we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. Twenty studies were eligible of which four were newly identified in this update. Only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Fifteen of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies.

Conclusions: This update does not change the conclusions of the review in which 16 studies were included. Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.

Conflict of interest statement

Competing Interests: The authors have declared that no competing interests exist.

Figures

Figure 1
Figure 1. Study flow diagram.
Figure 2
Figure 2. PRISMA flow diagram.
Figure 3
Figure 3. Status of approved protocols for Chan 2004b study .
Figure 4
Figure 4. Status of approved protocols for Easterbrook 1991 study .
Figure 5
Figure 5. Status of approved protocols for Dickersin 1992 study .
Figure 6
Figure 6. Status of approved protocols for Dickersin 1993 study .
Figure 7
Figure 7. Status of approved protocols for Stern 1997 study .
Figure 8
Figure 8. Status of approved protocols for Cooper 1997 study .
Figure 9
Figure 9. Status of trials for Wormald 1997 study .
Figure 10
Figure 10. Status of approved protocols for Ioannidis 1998 study .
Figure 11
Figure 11. Status of approved protocols for Pich 2003 study .
Figure 12
Figure 12. Status of approved protocols for Cronin 2004 study .
Figure 13
Figure 13. Status of approved protocols for Decullier 2005 study .
Figure 14
Figure 14. Status of approved protocols for Decullier 2006 study .
Figure 15
Figure 15. Status of approved protocols for Hahn 2002 study .
Figure 16
Figure 16. Status of approved protocols for Chan 2004a study .
Figure 17
Figure 17. Status of approved protocols for Ghersi 2006 study .
Figure 18
Figure 18. Status of approved protocols for von Elm 2008 study .
Figure 19
Figure 19. Status of approved protocols for Turer 2007 study .
Figure 20
Figure 20. Status of approved protocols for De Jong 2010 study .
Figure 21
Figure 21. Status of approved protocols for Blumle 2008 study .
Figure 22
Figure 22. Status of approved protocols for Hall 2007 study .

Similar articles

See all similar articles

Cited by 202 articles

See all "Cited by" articles

References

    1. Song F, Parekh S, Hooper L, Loke YK, Ryder J, et al. .. (2010) Dissemination and publication of research findings: an updated review of related biases.. Health Technol Assess 14. - PubMed
    1. Rothstein HR, Sutton AJ, Borenstein M (2005) Publication Bias in Meta-Analysis. Prevention, Assessment and Adjustments: Wiley.
    1. Dickersin K, Min YI (1993) NIH clinical trials and publication bias. Online J Curr Clin Trials Doc No 50. - PubMed
    1. Stern JM, Simes RJ (1997) Publication bias: evidence of delayed publication in a cohort study of clinical research projects. BMJ 315: 640–645. - PMC - PubMed
    1. Ioannidis JP (1998) Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 279: 281–286. - PubMed

MeSH terms

Feedback