Veterans Affairs intensive care unit risk adjustment model: validation, updating, recalibration

Crit Care Med. 2008 Apr;36(4):1031-42. doi: 10.1097/CCM.0b013e318169f290.

Abstract

Background: A valid metric is critical to measure and report intensive care unit (ICU) outcomes and drive innovation in a national system.

Objectives: To update and validate the Veterans Affairs (VA) ICU severity measure (VA ICU).

Research design: A validated logistic regression model was applied to two VA hospital data sets: 36,240 consecutive ICU admissions to a stratified random sample of moderate and large hospitals in 1999-2000 (cohort 1) and 81,964 cases from 42 VA Medical Centers in fiscal years 2002-2004 (cohort 2). The model was updated by adding diagnostic groups and expanding the source of admission variables.

Measures: C statistic, Hosmer-Lemeshow goodness-of-fit statistic, and Brier's score measured predictive validity. Coefficients from the 1997 model were applied to predictors (fixed) in a logistic regression model. A 10 x 10 table compared cases with both VA ICU and National Surgical Quality Improvement Performance metrics. The standardized mortality ratios divided observed deaths by the sum of predicted mortality.

Results: The fixed model in both cohorts had predictive validity (cohort 1: C statistic = 0.874, Hosmer-Lemeshow goodness-of-fit C statistic chi-square = 72.5; cohort 2: 0.876, 307), as did the updated model (cohort 2: C statistic = 0.887, Hosmer-Lemeshow goodness-of-fit C statistic chi-square = 39). In 7,411 cases with predictions in both systems, the standardized mortality ratio was similar (1.04 for VA ICU, 1.15 for National Surgical Quality Improvement Performance), and 92% of cases matched (+/-1 decile) when ordered by deciles of mortality. The VA ICU standardized mortality ratio correlates with the National Surgical Quality Improvement Performance standardized mortality ratio (r2 = .74). Variation in discharge and laboratory practices may affect performance measurement.

Conclusion: The VA ICU severity model has face, construct, and predictive validity.

Publication types

  • Multicenter Study
  • Research Support, U.S. Gov't, Non-P.H.S.

MeSH terms

  • Adult
  • Aged
  • Aged, 80 and over
  • Female
  • Hospital Mortality*
  • Hospitals, Veterans*
  • Humans
  • Intensive Care Units / statistics & numerical data*
  • Logistic Models
  • Male
  • Middle Aged
  • Patient Readmission / statistics & numerical data
  • Retrospective Studies
  • Risk Adjustment / methods*
  • Severity of Illness Index
  • United States