Comparison of 30-day mortality models for profiling hospital performance in acute ischemic stroke with vs without adjustment for stroke severity

JAMA. 2012 Jul 18;308(3):257-64. doi: 10.1001/jama.2012.7870.


Context: There is increasing interest in reporting risk-standardized outcomes for Medicare beneficiaries hospitalized with acute ischemic stroke, but whether it is necessary to include adjustment for initial stroke severity has not been well studied.

Objective: To evaluate the degree to which hospital outcome ratings and potential eligibility for financial incentives are altered after including initial stroke severity in a claims-based risk model for hospital 30-day mortality for acute ischemic stroke.

Design, setting, and patients: Data were analyzed from 782 Get With The Guidelines-Stroke participating hospitals on 127,950 fee-for-service Medicare beneficiaries with ischemic stroke who had a score documented for the National Institutes of Health Stroke Scale (NIHSS, a 15-item neurological examination scale with scores from 0 to 42, with higher scores indicating more severe stroke) between April 2003 and December 2009. Performance of claims-based hospital mortality risk models with and without inclusion of NIHSS scores for 30-day mortality was evaluated and hospital rankings from both models were compared.

Main outcomes measures: Model discrimination, hospital 30-day mortality outcome rankings, and value-based purchasing financial incentive categories.

Results: Across the study population, the mean (SD) NIHSS score was 8.23 (8.11) (median, 5; interquartile range, 2-12). There were 18,186 deaths (14.5%) within the first 30 days, including 7430 deaths (5.8%) during the index hospitalization. The hospital mortality model with NIHSS scores had significantly better discrimination than the model without (C statistic, 0.864; 95% CI, 0.861-0.867, vs 0.772; 95% CI, 0.769-0.776; P < .001). Among hospitals ranked in the top 20% or bottom 20% of performers by the claims model without NIHSS scores, 26.3% were ranked differently by the model with NIHSS scores. Of hospitals initially classified as having "worse than expected" mortality, 57.7% were reclassified to "as expected" by the model with NIHSS scores. The net reclassification improvement (93.1%; 95% CI, 91.6%-94.6%; P < .001) and integrated discrimination improvement (15.0%; 95% CI, 14.6%-15.3%; P < .001) indexes both demonstrated significant enhancement of model performance after the addition of NIHSS. Explained variance and model calibration was also improved with the addition of NIHSS scores.

Conclusion: Adding stroke severity as measured by the NIHSS to a hospital 30-day risk model based on claims data for Medicare beneficiaries with acute ischemic stroke was associated with considerably improved model discrimination and change in mortality performance rankings for a substantial portion of hospitals.

Publication types

  • Comparative Study
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Acute Disease
  • Aged
  • Aged, 80 and over
  • Brain Ischemia / classification
  • Brain Ischemia / mortality*
  • Female
  • Forecasting
  • Hospital Mortality / trends*
  • Hospitals / classification
  • Hospitals / standards*
  • Hospitals / statistics & numerical data
  • Humans
  • Insurance Claim Review
  • Male
  • Medicare / statistics & numerical data
  • Models, Theoretical
  • Outcome Assessment, Health Care / methods*
  • Quality Indicators, Health Care / statistics & numerical data*
  • Reimbursement, Incentive
  • Risk
  • Severity of Illness Index*
  • Stroke / classification
  • Stroke / mortality*
  • United States / epidemiology