Expert consensus vs empirical estimation of injury severity: effect on quality measurement in trauma

Abstract

Objective: To determine the extent to which the Injury Severity Score (ISS) and Trauma Mortality Probability Model (T-MPM), a new trauma injury score based on empirical injury severity estimates, agree on hospital quality.

Design, setting, and patients: Retrospective cohort study based on 66,214 patients in 68 hospitals. Four risk-adjustment models based on either ISS or T-MPM were constructed, with or without physiologic information.

Main outcome measures: Hospital quality was measured using the ratio of the observed-to-expected mortality rates. Pairwise comparisons of hospital quality based on ISSaugmented vs T-MPMaugmented were performed using the intraclass correlation coefficient and the kappa statistic.

Results: There was almost perfect agreement for the ratios of the observed to expected mortality rates based on the T-MPM vs the ISS when physiologic information was included in the model (intraclass correlation coefficient, 0.93). There was substantial agreement on which hospitals were identified as high-, intermediate-, and low-quality hospitals (kappa = 0.79). Excluding physiologic information decreased the level of agreement between the T-MPM and the ISS (intraclass correlation coefficient, 0.88 and kappa = 0.58).

Conclusions: The choice of expert-based or empirical Abbreviated Injury Score severity scores for individual injuries does not seem to have a significant effect on hospital quality measurement when physiologic information is included in the prediction model. This finding should help to convince all stakeholders that the quality of trauma care can be accurately measured and has face validity.