Magnitude of effects in clinical trials published in high-impact general medical journals

Int J Epidemiol. 2011 Oct;40(5):1280-91. doi: 10.1093/ije/dyr095. Epub 2011 Sep 8.


Background: Prestigious journals select for publication studies that are considered most important and informative. We aimed to examine whether high-impact general (HIG) medical journals systematically demonstrate more favourable results for experimental interventions compared with the rest of the literature.

Methods: We scrutinized systematic reviews of the Cochrane Database (Issue 4, 2009) and meta-analyses published in four general journals (2008-09). Eligible articles included ≥1 binary outcome meta-analysis(es) pertaining to effectiveness with ≥1 clinical trial(s) published in NEJM, JAMA or Lancet. Effect sizes in trials from NEJM, JAMA or Lancet were compared with those from other trials in the same meta-analyses by deriving summary relative odds ratios (sRORs). Additional analyses examined separately early- and late-published trials in HIG journals and journal-specific effects.

Results: A total of 79 meta-analyses including 1043 clinical trials were analysed. Trials in HIG journals had similar effects to trials in other journals, when there was large-scale evidence, but showed more favourable results for experimental interventions when they were small. When HIG trials had less than 40 events, the sROR was 1.64 [95% confidence interval (95% CI): 1.23-2.18). The difference was most prominent when small early trials published in HIG journals were compared with subsequent trials [sROR 2.68 (95% CI: 1.33-5.38)]. Late-published HIG trials showed no consistent inflation of effects. The patterns did not differ beyond chance between NEJM, JAMA or Lancet.

Conclusions: Small trials published in the most prestigious journals show more favourable effects for experimental interventions, and this is most prominent for early-published trials in such journals. No effect inflation is seen for large trials.

MeSH terms

  • Analysis of Variance
  • Bibliometrics*
  • Clinical Trials as Topic*
  • Humans
  • Meta-Analysis as Topic
  • Periodicals as Topic*
  • Publication Bias
  • Treatment Outcome