Risk of bias assessments for blinding of participants and personnel in Cochrane reviews were frequently inadequate

Abstract

Objectives: The objective of this study was to analyze adequacy of judgments about risk of bias (RoB) for blinding of participants and personnel (performance bias) in Cochrane systematic reviews of randomized controlled trials (RCTs).

Study design and setting: We extracted judgments and supporting comments for performance bias from Cochrane reviews' RoB tables using automated data scraping. We parsed all intervention descriptions, judgments about risk of performance bias, and comments supporting judgments into simple categories. We assessed adequacy of RoB judgments against recommendations from the Cochrane Handbook.

Results: We analyzed judgments for performance bias of 10,429 RCTs included in 718 Cochrane reviews. Overall, 1,828 out of 6,918 judgments (26%) for performance bias were not in line with the Cochrane Handbook and were therefore considered inadequate. In reviews where Cochrane authors have split the performance bias domain into two subdomains, based on blinded individuals, we found lower prevalence of inadequate risk of bias judgments, with 9% of judgments for blinding of participants, and 5.8% judgments for the blinding of personnel subdomain being judged inadequately.

Conclusion: In Cochrane reviews, risk of bias assessments for blinding of participants and personnel were frequently not in line with Cochrane Handbook recommendations. Interventions to improve these assessments should be taken into consideration.

Keywords: Blinding; Cochrane; Participants; Performance bias; Personnel; Risk of bias; Systematic reviews.