Objective: To examine the validity of hybrid quality measures that use both clinical registry and administrative claims data, capitalizing on the strengths of each data source.
Background: Previous studies demonstrate substantial disagreement between clinical registry and administrative claims data on the occurrence of postoperative complications. Clinical data have greater validity than claims data for quality measurement but can be burdensome for hospitals to collect.
Methods: American College of Surgeons National Surgical Quality Improvement Program records were linked to Medicare inpatient claims (2005-2008). National Quality Forum-endorsed risk-adjusted measures of 30-day postoperative complications or death assessed hospital quality for patients undergoing colectomy, lower extremity bypass, or all surgical procedures. Measures use hierarchical multivariable logistic regression to identify statistical outliers. Measures were applied using clinical data, claims data, or a hybrid of both data sources. Kappa statistics assessed agreement on determinations of hospital quality.
Results: A total of 111,984 patients participated from 206 hospitals. Agreement on hospital quality between clinical and claims data was poor. Hybrid models using claims data to risk-adjust complications identified by clinical data had moderate agreement with all clinical data models, whereas hybrid models using clinical data to risk-adjust complications identified by claims data had routinely poor agreement with all clinical data models.
Conclusions: Assessments of hospital quality differ substantially when using clinical registry versus administrative claims data. A hybrid approach using claims data for risk adjustment and clinical data for complications may be a valid alternative with lower data collection burden. For quality measures focused on postoperative complications to be meaningful, such policies should require, at a minimum, collection of clinical outcomes data.