Objective: To develop a new method to evaluate the performance of individual ICUs through the calculation and visualisation of risk profiles.
Methods: The study included 102,561 patients consecutively admitted to 77 ICUs in Austria. We customized the function which predicts hospital mortality (using SAPS II) for each ICU. We then compared the risks of hospital mortality resulting from this function with the risks which would be obtained using the original function. The derived risk ratio was then plotted together with point-wise confidence intervals in order to visualise the individual risk profile of each ICU over the whole spectrum of expected hospital mortality.
Main measurements and results: We calculated risk profiles for all ICUs in the ASDI data set according to the proposed method. We show examples how the clinical performance of ICUs may depend on the severity of illness of their patients. Both the distribution of the Hosmer-Lemeshow goodness-of-fit test statistics and the histogram of the corresponding P values demonstrated a good fit of the individual risk models.
Conclusions: Our risk profile model makes it possible to evaluate ICUs on the basis of the specific risk for patients to die compared to a reference sample over the whole spectrum of hospital mortality. Thus, ICUs at different levels of severity of illness can be directly compared, giving a clear advantage over the use of the conventional single point estimate of the overall observed-to-expected mortality ratio.