Empirical performance of the case-control method: lessons for developing a risk identification and analysis system

Drug Saf. 2013 Oct;36 Suppl 1:S73-82. doi: 10.1007/s40264-013-0105-z.


Background: Considerable attention now focuses on the use of large-scale observational healthcare data for understanding drug safety. In this context, analysts utilize a variety of statistical and epidemiological approaches such as case-control, cohort, and self-controlled methods. The operating characteristics of these methods are poorly understood.

Objective: Establish the operating characteristics of the case-control method for large scale observational analysis in drug safety.

Research design: We empirically evaluated the case-control approach in 5 real observational healthcare databases and 6 simulated datasets. We retrospectively studied the predictive accuracy of the method when applied to a collection of 165 positive controls and 234 negative controls across 4 outcomes: acute liver injury, acute myocardial infarction, acute kidney injury, and upper gastrointestinal bleeding.

Results: In our experiment, the case-control method provided weak discrimination between positive and negative controls. Furthermore, the method yielded positively biased estimates and confidence intervals that had poor coverage properties.

Conclusions: For the four outcomes we examined, the case-control method may not be the method of choice for estimating potentially harmful effects of drugs.

MeSH terms

  • Acute Kidney Injury / chemically induced
  • Area Under Curve
  • Case-Control Studies*
  • Chemical and Drug Induced Liver Injury / diagnosis
  • Drug-Related Side Effects and Adverse Reactions / diagnosis*
  • Gastrointestinal Hemorrhage / chemically induced
  • Humans
  • Myocardial Infarction / chemically induced
  • Probability
  • Research Design*
  • Retrospective Studies
  • Risk Assessment / methods*