Utility of automated data-adaptive propensity score method for confounding by indication in comparative effectiveness study in real world Medicare and registry data

PLoS One. 2022 Aug 15;17(8):e0272975. doi: 10.1371/journal.pone.0272975. eCollection 2022.


Background: Confounding by indication is a serious threat to comparative studies using real world data. We assessed the utility of automated data-adaptive analytic approach for confounding adjustment when both claims and clinical registry data are available.

Methods: We used a comparative study example of carotid artery stenting (CAS) vs. carotid endarterectomy (CEA) in 2005-2008 when CAS was only indicated for patients with high surgical risk. We included Medicare beneficiaries linked to the Society for Vascular Surgery's Vascular Registry >65 years old undergoing CAS/CEA. We compared hazard ratios (HRs) for death while adjusting for confounding by combining various 1) Propensity score (PS) modeling strategies (investigator-specified [IS-PS] vs. automated data-adaptive [ada-PS]); 2) data sources (claims-only, registry-only and claims-plus-registry); and 3) PS adjustment approaches (matching vs. quintiles-adjustment with/without trimming). An HR of 1.0 was used as a benchmark effect estimate based on CREST trial.

Results: The cohort included 1,999 CAS and 3,255 CEA patients (mean age 76). CAS patients were more likely symptomatic and at high surgical risk, and experienced higher mortality (crude HR = 1.82 for CAS vs. CEA). HRs from PS-quintile adjustment without trimming were 1.48 and 1.52 for claims-only IS-PS and ada-PS, 1.51 and 1.42 for registry-only IS-PS and ada-PS, and 1.34 and 1.23 for claims-plus-registry IS-PS and ada-PS, respectively. Estimates from other PS adjustment approaches showed similar patterns.

Conclusions: In a comparative effectiveness study of CAS vs. CEA with strong confounding by indication, ada-PS performed better than IS-PS in general, but both claims and registry data were needed to adequately control for bias.

Publication types

  • Research Support, U.S. Gov't, P.H.S.

MeSH terms

  • Aged
  • Carotid Stenosis* / surgery
  • Comparative Effectiveness Research
  • Endarterectomy, Carotid*
  • Humans
  • Medicare
  • Propensity Score
  • Registries
  • Risk Assessment
  • Risk Factors
  • Stents
  • Stroke*
  • Time Factors
  • Treatment Outcome
  • United States

Grant support

This project is funded by contract HHSA290-2005-0016-I TO3 from the Agency for Healthcare Research and Quality (AHRQ), U.S. Department of Health and Human Services (DHHS) as part of the Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) program; and by IAA Contract 500-2010-00001I TO6 and CEA Contract 500-2010-00001I TO2 from the Centers for Medicare & Medicaid Services (CMS), U.S. DHHS. The funding agency had no role in the design and conduct of the study or in the collection, analysis, and interpretation of the data.