High quality of the evidence for medical and other health-related interventions was uncommon in Cochrane systematic reviews

Padhraig S Fleming; Despina Koletsi; John P A Ioannidis; Nikolaos Pandis

doi:10.1016/j.jclinepi.2016.03.012

High quality of the evidence for medical and other health-related interventions was uncommon in Cochrane systematic reviews

J Clin Epidemiol. 2016 Oct:78:34-42. doi: 10.1016/j.jclinepi.2016.03.012. Epub 2016 Mar 29.

Authors

Padhraig S Fleming¹, Despina Koletsi², John P A Ioannidis³, Nikolaos Pandis⁴

Affiliations

¹ Oral Growth and Development, Institute of Dentistry, Barts and The London School of Medicine and Dentistry, Queen Mary University of London, London E1 2AD, England. Electronic address: padhraig.fleming@gmail.com.
² Department of Orthodontics, University of Athens, 2 Thivon Street, 115 27, Goudi, Athens, Greece.
³ Department of Medicine, Stanford Prevention Research Center, Stanford University School of Medicine, Stanford, CA, USA; Department of Health Research and Policy, Stanford University School of Medicine, Stanford, CA, USA; Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, CA, USA; Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA.
⁴ Department of Orthodontics and Dentofacial Orthopedics, Dental School, Medical Faculty, University of Bern, Hochschulstrasse 4, 3012 Bern, Switzerland; Private Practice, 29 P. Zafiropoulou Street, Corfu 49100, Greece.

PMID: 27032875
DOI: 10.1016/j.jclinepi.2016.03.012

Abstract

Objectives: To appraise the quality of evidence in systematic reviews (SRs) within the Cochrane Database of Systematic Reviews (CDSRs) across diverse topics and to explore the relationship between the strength of evidence using Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) and the probability that authors would interpret that an intervention may be of value.

Study design and setting: We evaluated the SRs published on the CDSR from January 1, 2013, to June 30, 2014. Two authors identified relevant SRs by independent searching of the Cochrane register. We further focused on SRs that incorporated tables with GRADE [summary of findings (SoF)]. Data were extracted independently by two authors. The quality of the evidence for the first listed primary outcome in SoF tables in each review and reasons for upgrade or downgrade were recorded.

Results: Overall, 1,394 SRs were identified. Of these, 608 (43.6%) incorporated GRADE. Within these reviews, only 13.5% (n = 82) reported a high quality and 30.8% (n = 187) a moderate quality of evidence for the first listed primary outcome, whereas 31.7% (n = 193) had low level and 24% (n = 146) had very low level of evidence. High quality of evidence was more common in updated compared to new reviews and in pharmacologic than other types of interventions. Even when all outcomes listed in the SoFs were considered, only 116/608 (19.1%) of SRs had at least one outcome with high quality of evidence. Overall, only 4.1% (25/608) of SRs incorporating GRADE in SoF tables had high quality of evidence, allied both to significant results and a favorable interpretation of the intervention by the reviewers.

Conclusion: Evidence of high quality is uncommon for medical and health-related interventions assessed with GRADE within the CDSR, and favorable evidence of high quality is even more uncommon.

Keywords: GRADE; Quality evidence; Randomized controlled trial; Systematic review; Treatment recommendation; Cochrane.

Publication types

Review
Systematic Review

MeSH terms

Epidemiologic Studies*
Guidelines as Topic / standards*
Humans
Outcome Assessment, Health Care / methods
Outcome Assessment, Health Care / standards*
Outcome Assessment, Health Care / statistics & numerical data*
Review Literature as Topic*