Reliability of hospital readmission rates in vascular surgery

Abstract

Objective: The Center for Medicare and Medicaid Services recently began assessing financial penalties to hospitals with high readmission rates for a narrow set of medical conditions. Because these penalties will be extended to surgical conditions in the near future, we sought to determine whether readmissions are a reliable predictor of hospital performance with vascular surgery.

Methods: We examined 4 years of national Medicare claims data from 1576 hospitals on beneficiaries undergoing three common vascular procedures: open or endovascular abdominal aortic aneurysm repair (n = 81,520) or lower extremity arterial bypass (n = 57,190). First, we divided our population into two groups on the basis of operative date (2005-2006 and 2007-2008) and generated hospital risk- and reliability-adjusted readmission rates for each time period. We evaluated reliability through the use of the "test-retest" method; highly reliable measures will show little variation in rates over time. Specifically, we evaluated the year-to-year reliability of readmissions by calculating Spearman rank correlation and weighted κ tests for readmission rates between the two time periods.

Results: The Spearman coefficient between 2005-2006 readmissions rankings and 2007-2008 readmissions rankings was 0.57 (P < .001) and weighted κ was 0.42 (P < .001), indicating a moderate correlation. However, only 32% of the variation in hospital readmission rates in 2007-2008 was explained by readmissions during the 2 prior years. There were major reclassifications of hospital rankings between years, with 63% of hospitals migrating among performance quintiles between 2005-2006 and 2007-2008.

Conclusions: Risk-adjusted readmission rates for vascular surgery vary substantially year to year; this implies that much of the observed variation in readmission rates is either random or caused by unmeasured factors and not caused by changes in hospital quality that may be captured by administrative data.

Figure 1

Hospital rankings based on risk- and reliability-adjusted readmission rates in 2005-2006 versus 2007-2008, stratified by operation type. Spearman’s correlation coefficients and associate p values are listed for each operation.

Figure 2

Comparison of Quintile Rankings Between 2005-06 and 2007-08 for 1,576 hospitals. Shading intensity represents the magnitude reclassification. (n) represents the number of hospitals in each quintile combination. Overall 63% of hospitals were reclassified between the two year groups.

Figure 3

Risk- and reliability-adjusted hospital readmission rates in 2007-08, according to quintiles of historical readmission performance in 2005-06. Both historical and subsequent readmission rates are adjusted for patient characteristics.