The use of propensity scores and observational data to estimate randomized controlled trial generalizability bias

Stat Med. 2013 Sep 10;32(20):3552-68. doi: 10.1002/sim.5802. Epub 2013 Apr 1.


Although randomized controlled trials are considered the 'gold standard' for clinical studies, the use of exclusion criteria may impact the external validity of the results. It is unknown whether estimators of effect size are biased by excluding a portion of the target population from enrollment. We propose to use observational data to estimate the bias due to enrollment restrictions, which we term generalizability bias. In this paper, we introduce a class of estimators for the generalizability bias and use simulation to study its properties in the presence of non-constant treatment effects. We find the surprising result that our estimators can be unbiased for the true generalizability bias even when all potentially confounding variables are not measured. In addition, our proposed doubly robust estimator performs well even for mis-specified models.

Keywords: causal effect; observational studies; propensity score; randomized controlled trials; sample selection error.

Publication types

  • Research Support, N.I.H., Extramural

MeSH terms

  • Bias*
  • Computer Simulation
  • Confounding Factors, Epidemiologic*
  • Data Interpretation, Statistical*
  • Extraction, Obstetrical / instrumentation
  • Female
  • Humans
  • Models, Statistical*
  • Propensity Score*
  • Randomized Controlled Trials as Topic / methods*
  • Treatment Outcome