External validation of the SAPS II, APACHE II and APACHE III prognostic models in South England: a multicentre study

Intensive Care Med. 2003 Feb;29(2):249-56. doi: 10.1007/s00134-002-1607-9. Epub 2003 Jan 18.


Objective: External validation of three prognostic models in adult intensive care patients in South England. DESIGN. Prospective cohort study.

Setting: Seventeen intensive care units (ICU) in the South West Thames Region in South England.

Patients and participants: Data of 16646 patients were analysed.

Interventions: None.

Measurements and results: We compared directly the predictive accuracy of three prognostic models (SAPS II, APACHE II and III), using formal tests of calibration and discrimination. The external validation showed a similar pattern for all three models tested: good discrimination, but imperfect calibration. The areas under the receiver operating characteristics (ROC) curves, used to test discrimination, were 0.835 and 0.867 for APACHE II and III, and 0.852 for the SAPS II model. Model calibration was assessed by Lemeshow-Hosmer C-statistics and was Chi(2 )=232.1 for APACHE II, Chi(2 )=443.3 for APACHE III and Chi(2 )=287.5 for SAPS II.

Conclusions: Disparity in case mix, a higher prevalence of outcome events and important unmeasured patient mix factors are possible sources for the decay of the models' predictive accuracy in our population. The lack of generalisability of standard prognostic models requires their validation and re-calibration before they can be applied with confidence to new populations. Customisation of existing models may become an important strategy to obtain authentic information on disease severity, which is a prerequisite for reliably measuring and comparing the quality and cost of intensive care.

Publication types

  • Comparative Study
  • Multicenter Study
  • Research Support, Non-U.S. Gov't
  • Validation Study

MeSH terms

  • Adolescent
  • Adult
  • Aged
  • Aged, 80 and over
  • Calibration
  • Diagnosis-Related Groups / classification
  • Diagnosis-Related Groups / statistics & numerical data
  • Discriminant Analysis
  • England / epidemiology
  • Female
  • Hospital Costs / statistics & numerical data
  • Hospital Mortality*
  • Humans
  • Intensive Care Units / economics
  • Intensive Care Units / standards
  • Intensive Care Units / statistics & numerical data*
  • Male
  • Middle Aged
  • Models, Statistical*
  • Predictive Value of Tests
  • Probability
  • Prognosis
  • Prospective Studies
  • Quality of Health Care
  • ROC Curve
  • Risk Factors
  • Severity of Illness Index*