Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Comparative Study
. 2014 Dec 15;14:116.
doi: 10.1186/1471-2253-14-116. eCollection 2014.

External Validation of the Intensive Care National Audit & Research Centre (ICNARC) Risk Prediction Model in Critical Care Units in Scotland

Affiliations
Free PMC article
Comparative Study

External Validation of the Intensive Care National Audit & Research Centre (ICNARC) Risk Prediction Model in Critical Care Units in Scotland

David A Harrison et al. BMC Anesthesiol. .
Free PMC article

Abstract

Background: Risk prediction models are used in critical care for risk stratification, summarising and communicating risk, supporting clinical decision-making and benchmarking performance. However, they require validation before they can be used with confidence, ideally using independently collected data from a different source to that used to develop the model. The aim of this study was to validate the Intensive Care National Audit & Research Centre (ICNARC) model using independently collected data from critical care units in Scotland.

Methods: Data were extracted from the Scottish Intensive Care Society Audit Group (SICSAG) database for the years 2007 to 2009. Recoding and mapping of variables was performed, as required, to apply the ICNARC model (2009 recalibration) to the SICSAG data using standard computer algorithms. The performance of the ICNARC model was assessed for discrimination, calibration and overall fit and compared with that of the Acute Physiology And Chronic Health Evaluation (APACHE) II model.

Results: There were 29,626 admissions to 24 adult, general critical care units in Scotland between 1 January 2007 and 31 December 2009. After exclusions, 23,269 admissions were included in the analysis. The ICNARC model outperformed APACHE II on measures of discrimination (c index 0.848 versus 0.806), calibration (Hosmer-Lemeshow chi-squared statistic 18.8 versus 214) and overall fit (Brier's score 0.140 versus 0.157; Shapiro's R 0.652 versus 0.621). Model performance was consistent across the three years studied.

Conclusions: The ICNARC model performed well when validated in an external population to that in which it was developed, using independently collected data.

Keywords: Critical care; Intensive care units; Models; Prognosis; Risk adjustment; Severity of illness index; Statistical; Validation studies.

Figures

Figure 1
Figure 1
Distribution of predicted risk. Distribution of predicted risk from the ICNARC risk prediction model (2009 recalibration) among 23,269 admissions to adult, general critical care units in Scotland.
Figure 2
Figure 2
Receiver operating characteristic curves. Receiver operating characteristic (ROC) curves for the ICNARC (2009 recalibration) and APACHE II risk prediction models among 23,269 admissions to adult, general critical care units in Scotland.
Figure 3
Figure 3
Calibration plots. Calibration plots showing observed against expected mortality in ten equal sized groups for the ICNARC (2009 recalibration) and APACHE II risk prediction models among 23,269 admissions to adult, general critical care units in Scotland.

Similar articles

See all similar articles

Cited by 5 articles

References

    1. Higgins TL. Quantifying risk and benchmarking performance in the adult intensive care unit. J Intensive Care Med. 2007;22:141–156. doi: 10.1177/0885066607299520. - DOI - PubMed
    1. Turner EL, Perel P, Clayton T, Edwards P, Hernández AV, Roberts I, Shakur H, Steyerberg EW, CRASH Trial Collaborators Covariate adjustment increased power in randomized controlled trials: an example in traumatic brain injury. J Clin Epidemiol. 2012;65:474–481. doi: 10.1016/j.jclinepi.2011.08.012. - DOI - PMC - PubMed
    1. Wunsch H, Linde-Zwirble WT, Angus DC. Methods to adjust for bias and confounding in critical care health services research involving observational data. J Crit Care. 2006;21:1–7. doi: 10.1016/j.jcrc.2006.01.004. - DOI - PubMed
    1. Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med. 2000;19:453–473. doi: 10.1002/(SICI)1097-0258(20000229)19:4<453::AID-SIM350>3.0.CO;2-5. - DOI - PubMed
    1. Altman DG, Vergouwe Y, Royston P, Moons KGM. Prognosis and prognostic research: validating a prognostic model. BMJ. 2009;338:b605. doi: 10.1136/bmj.b605. - DOI - PubMed
Pre-publication history
    1. The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2253/14/116/prepub

Publication types

Feedback