Many meta-analyses of rare events in the Cochrane Database of Systematic Reviews were underpowered

Abstract

Background and objective: Meta-analysis is a statistical method with the ability to increase the power for statistical inference, while it may still face the problem of being underpowered. In this study, we investigated the power to detect certain true effects for published meta-analyses of rare events.

Methods: We extracted data from the Cochrane Database of Systematic Reviews for meta-analyses of rare events from January 2003 to May 2018. We retrospectively estimated the power to detect a 10-50% relative risk reduction (RRR) of eligible meta-analyses. The proportion of meta-analyses achieved a sufficient power (≥0.8) were estimated.

Results: We identified 4,177 meta-analyses. The median power to detect 10%, 30%, and 50% RRR were 0.06 (interquartile range [IQR]: 0.05 to 0.06), 0.08 (IQR: 0.06 to 0.15), and 0.17 (IQR: 0.10 to 0.42), respectively); the corresponding proportion of meta-analyses that reached sufficient power were 0.32%, 3.68%, and 11.81%. Meta-analyses incorporating data from more studies had higher probability to achieve a sufficient power (rate ratio = 2.49, 95% CI: 1.76, 3.52, P < 0.001).

Conclusion: Most of the meta-analyses of rare events in Cochrane systematic reviews were underpowered. Future meta-analysis of rare events should report the power of the results to support informative conclusions.

Keywords: Cochrane systematic reviews; Decision-making; Meta-analysis of rare events; Power analysis; Relative risk redcution; Statistical inference.