Performance of propensity score methods when comparison groups originate from different data sources

Abstract

Purpose: To examine the performance of propensity score-based methods for estimating relative risks when exposed and comparison subjects are selected from different data sources.

Methods: We conducted Monte Carlo simulations to assess the performance of propensity score methods under various scenarios in which exposed and comparison subjects were selected from different data sources for a comparative effectiveness study of a medical device.

Results: The use of propensity score methods in our simulated data scenarios often yielded estimates of relative risk that were close to the true effect, unless the comparison group differed from the exposed group systematically on a factor associated with the outcome. This situation caused severe bias regardless of which method was used but could be overcome if the exposed group could be restricted similarly to the comparison group. Mean square error of relative risk estimates was lowest for similarly restricted study groups and when the comparison group could be considered a random sample of the source population that generated the exposed group.

Conclusions: When exposed and comparison groups originated from different data sources, all propensity score methods yielded relatively unbiased and consistent estimates of relative risk in most situations reflected in our simulation study.