Profiling hospitals on bariatric surgery quality: which outcomes are most reliable?

Abstract

Background: Under the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program, hospitals will receive risk-adjusted outcomes feedback for peer comparisons and benchmarking. It remains uncertain whether bariatric outcomes have adequate reliability to identify outlying performance, especially for hospitals with low caseloads that will be included in the program. We explored the ability of risk-adjusted outcomes to identify outlying hospital performance with bariatric surgery for a range of hospital caseloads.

Study design: We used the 2010 State Inpatient Databases for 12 states (N = 31,240 patients) to assess different outcomes (eg, complications, reoperation, and mortality) after bariatric stapling procedures. We first quantified outcomes reliability on a 0 (no reliability) to 1 (perfect reliability) scale. We then assessed whether risk- and reliability-adjusted outcomes could identify outlying performance among hospitals with different annual caseloads.

Results: Overall and serious complications had the highest overall reliability, but this was heavily dependent on caseload. For example, among hospitals with the lowest caseloads (mean 56 cases/year), reliability for overall complications was 0.49 and 6.0% of hospitals had outlying performance. For hospitals with the highest caseloads (mean 298 cases/year), reliability for overall complications was 0.79 and 30.3% of hospitals had outlying performance. Reoperation had adequate reliability for hospitals with caseloads higher than 120 cases/year. Mortality had unacceptably low reliability regardless of hospital caseloads.

Conclusions: Overall complications and serious complications have adequate reliability for distinguishing outlying performance with bariatric surgery, even for hospitals with low annual caseloads. Rare outcomes, such as reoperations, have inadequate reliability to inform peer-based comparisons for hospitals with low annual caseloads, and mortality has unacceptably low reliability for bariatric performance profiling.

Figure 1

Example performance report (any complication, hospitals with at least 125 cases/yr, laparoscopic gastric bypass procedures). Diamonds: hospital risk adjusted outcome rates with 95% confidence intervals. Green: low outliers, have 95% confidence intervals less than the average outcomes rate. Red: high outliers, have 95% confidence intervals greater than the average outcomes rate. Solid horizontal line, overall average outcomes rate.

Figure 2

(A-C) Risk-and reliability adjusted outcome rates across hospital caseload terciles (serious complications). Y-axis, percent events (%), X-axis, hospital rank. (A) Lowest caseloads (mean 56.4 cases/y); reliability 0.423. (B) Middle caseloads (mean 119.7 cases/y); reliability 0.484. (C) Highest caseloads (mean 298.2 cases/y); reliability 0.639. Y-axis scale is the same for all figures, to illustrate shrinking of 95% confidence intervals as caseloads.

Figure 3

Risk-and reliability adjusted outcome rates (serious complications) at different caseload thresholds for reporting. Y-axis: percent events (%), X-axis: hospital rank. (A) No caseload threshold; reliability 0.475. (B) Reporting threshold 50 cases/y; reliability 0.508. (C) Reporting threshold 100 cases/y; reliability 0.527. (D) Reporting threshold 125 cases/y; reliability 0.534.