Methods to Select Results to Include in Meta-Analyses Deserve More Consideration in Systematic Reviews

J Clin Epidemiol. 2015 Nov;68(11):1282-91. doi: 10.1016/j.jclinepi.2015.02.009. Epub 2015 Mar 6.


Objectives: To investigate how often systematic reviewers encounter multiple trial effect estimates that are available for inclusion in a particular meta-analysis (multiplicity of results) and the methods they use to select effect estimates.

Study design and setting: We randomly sampled Cochrane and MEDLINE-indexed non-Cochrane reviews published between January 2010 and January 2012. The first presented meta-analysis of an effect measure for a continuous outcome in each review was identified, and methods to select results to include in this meta-analysis were extracted from review protocols and reviews. All effect estimates that were available for inclusion in the meta-analyses were extracted from trial reports.

Results: We examined 44 reviews. Multiplicity of results was common, occurring in 49% of trial reports (n = 210). Prespecification of decision rules to select results from multiple measurement scales and intervention/control groups (in multi-arm trials) was uncommon (19% and 14% of 21 review protocols, respectively). Overall, 70% of reviews included at least one randomized controlled trial with multiplicity of results, but this occurred less frequently in reviews with a protocol (risk difference, -25%; 95% confidence interval: -52%, 1%).

Conclusion: Systematic reviewers are likely to encounter multiplicity of results in the included trials. We recommend that systematic reviewers always consider predefining methods to select results to include in meta-analyses. Methods focusing on selection of measurement scales and how to deal with multi-arm trials would be most valuable.

Keywords: Bias; Meta-analysis; Randomized controlled trials; Reporting; Research methodology; Systematic review.

MeSH terms

  • Information Storage and Retrieval
  • Meta-Analysis as Topic*
  • Research Design*
  • Review Literature as Topic*