Specific instructions for estimating unclearly reported blinding status in randomized trials were reliable and valid

J Clin Epidemiol. 2012 Mar;65(3):262-7. doi: 10.1016/j.jclinepi.2011.04.015. Epub 2011 Dec 24.

Abstract

Objective: To test the reliability and validity of specific instructions to classify blinding, when unclearly reported in randomized trials, as "probably done" or "probably not done."

Study design and setting: We assessed blinding of patients, health care providers, data collectors, outcome adjudicators, and data analysts in 233 randomized trials in duplicate and independently using detailed instructions. The response options were "definitely yes," "probably yes," "probably no," and "definitely no." We contacted authors for data verification (46% response). For each of the five questions, we assessed reliability by calculating the agreement between the two reviewers and validity by calculating the agreement between reviewers' consensus and verified data.

Results: The percentage with unclear blinding status varied between 48.5% (patients) and 84.1% (data analysts). Reliability was moderate for blinding of outcome adjudicators (κ=0.52) and data analysts (κ=0.42) and substantial for blinding of patients (κ=0.71), providers (κ=0.68), and data collectors (κ=0.65). The raw agreement between the consensus record and the author-verified record varied from 84.1% (blinding of data analysts) to 100% (blinding of health care providers).

Conclusion: With the possible exception of blinding of data analysts, use of "probably yes" and "probably no" instead of "unclear" may enhance the assessment of blinding in trials.

Publication types

  • Evaluation Study
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Algorithms
  • Bias
  • Consensus
  • Double-Blind Method
  • Epidemiologic Research Design*
  • Guideline Adherence
  • Guidelines as Topic*
  • Humans
  • Randomized Controlled Trials as Topic / methods*
  • Randomized Controlled Trials as Topic / standards*
  • Randomized Controlled Trials as Topic / statistics & numerical data
  • Research Design
  • Review Literature as Topic
  • Single-Blind Method