Evaluation of the Methods Used by Medicare's Hospital-Acquired Condition Reduction Program to Identify Outlier Hospitals for Surgical Site Infection

Abstract

Background: The Hospital Acquired Condition Reduction Program (HACRP) is a national pay-for-performance program that includes a measure of surgical site infection (SSI) after hysterectomy and colectomy. This study compares the HACRP SSI measure with other published methods.

Study design: This was a retrospective cohort study from the Michigan Surgical Quality Collaborative (MSQC). The outcome was 30-day, adjusted deep and organ space SSI ("complex SSI"). Observed-to-expected ratios of complex SSI for each hospital were calculated using HACRP, National Healthcare Safety Network (NHSN), and MSQC methodologies. C-statistics were compared between models. Hospital rankings were compared, and ladder plots show changes in hospitals' HACRP scores that derive from each algorithm.

Results: Complex SSI occurred in 1.1% (190 of 16,672) of hysterectomies and 4.8% (n = 514 of 10,725) of colectomies. The HACRP risk-adjustment model for hysterectomy had a C-statistic of 0.55, significantly lower than NHSN (0.61, p = 0.0461) or MSQC models (0.77, p < 0.0001). For colectomy, C-statistics were 0.57, 0.66 (p < 0.0001) and 0.73 (p < 0.0001), respectively. For both operations, there were 5 high-outlier hospitals using HACRP, but fewer (4 or 3) using the other methods. Most hospitals in the bottom quartile were not statistical outliers, but would be flagged under HACRP. More than 50% of hospitals changed ranking position between models, which would result in different scores under HACRP.

Conclusions: This study showed that the HACRP SSI measure unfairly places hospitals at risk for financial penalties that are not statistical outliers. Policy makers need to weigh the burden of data collection and the accuracy needed to identify hospitals for financial reward or penalty.