Confusion Instead of Clarity: Publicly Reported Cardiac Surgery Ratings for Coronary Artery Bypass Grafting and Aortic Valve Replacement

Abstract

Background: Public reporting of cardiac surgery ratings has been advocated to inform patient selection of hospitals. Although Society of Thoracic Surgeons (STS) ratings are based on audited risk-adjusted patient outcomes, other rating systems rely on administrative databases. In this study, we evaluate correlation among 4 widely used hospital rating systems for coronary artery bypass grafting (CABG) and aortic valve replacement (AVR).

Study design: We identified an initial cohort of 602 hospitals from US News & World Report's (USN) listing of the 2016-2017 "Best Hospitals for Cardiology & Heart Surgery." From this cohort, current publicly available CABG and AVR ratings were collected from the STS, USN, Centers for Medicare & Medicaid Services, and Healthgrades. All 4 rating systems rated hospitals as high, average, or below average performers for each procedure. We then determined the match rate between rating systems for individual hospitals and assessed interrater reliability with Cohen's κ.

Results: Rating systems had different distributions of high and low performing ratings assigned. USN rated hospitals as high performing for both CABG and AVR more frequently compared with STS, Healthgrades, and Centers for Medicare & Medicaid Services. For CABG, the match rate between systems varied from 50% to 85%, with the best match between STS and Centers for Medicare & Medicaid Services. Similarly for AVR, the match rate varied from 50% to 73%, with the best match between STS and Healthgrades. Interrater reliability was poor among the 4 rating systems (κ < 0.2) and consistent with no agreement for CABG and AVR ratings.

Conclusions: Publicly reported cardiac surgery ratings have significant discrepancy and poor correlation. This might confuse instead of clarify public perception of hospital quality for cardiac surgery.