Meta-analyses frequently pooled different study types together: a meta-epidemiological study

Abstract

Objective: To evaluate the characteristics of therapeutic meta-analyses including both observational studies and randomized controlled trials (RCTs), how these studies were combined and whether there were differences in treatment effects.

Study design and setting: Meta-epidemiological study of meta-analyses, including both observational studies and RCTs. We searched MEDLINE for the five leading journals of each medical category according to Journal Citation Reports) and Cochrane Database of Systematic Reviews, from 2014 to 2018 for eligible meta-analyses and extracted how observational studies and RCTs were combined and results for each study.

Results: Of the 102 included meta-analyses, observational studies and RCTs were combined together without a subgroup analysis in 39 (38%) and with subgroup analysis in 15 (15%); they were pooled separately for the same outcome in 11 (11%) and not for the same outcome in 9 (9%). In 28 (27%) meta-analyses, only RCTs were combined, with a qualitative description of observational studies. Treatment effect estimates did not differ between observational studies and RCTs (ratio of estimates = 0.98 [95% confidence interval 0.80-1.21]), with substantial heterogeneity (I² = 59%).

Conclusion: Many meta-analyses, including both observational studies and RCTs pool results from both study types. Although treatment effects did not differ between them on average, we identified situations for which estimates differed.

Keywords: Comparative effectiveness research; Meta-analysis; Meta-epidemiology; Observational studies; Randomized controlled trials; Systematic review; Therapeutic intervention.