Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score

Abstract

Objective: To develop and validate a pragmatic risk score to predict mortality in patients admitted to hospital with coronavirus disease 2019 (covid-19).

Design: Prospective observational cohort study.

Setting: International Severe Acute Respiratory and emerging Infections Consortium (ISARIC) World Health Organization (WHO) Clinical Characterisation Protocol UK (CCP-UK) study (performed by the ISARIC Coronavirus Clinical Characterisation Consortium-ISARIC-4C) in 260 hospitals across England, Scotland, and Wales. Model training was performed on a cohort of patients recruited between 6 February and 20 May 2020, with validation conducted on a second cohort of patients recruited after model development between 21 May and 29 June 2020. PARTICIPANTS: Adults (age ≥18 years) admitted to hospital with covid-19 at least four weeks before final data extraction.

Main outcome measure: In-hospital mortality.

Results: 35 463 patients were included in the derivation dataset (mortality rate 32.2%) and 22 361 in the validation dataset (mortality rate 30.1%). The final 4C Mortality Score included eight variables readily available at initial hospital assessment: age, sex, number of comorbidities, respiratory rate, peripheral oxygen saturation, level of consciousness, urea level, and C reactive protein (score range 0-21 points). The 4C Score showed high discrimination for mortality (derivation cohort: area under the receiver operating characteristic curve 0.79, 95% confidence interval 0.78 to 0.79; validation cohort: 0.77, 0.76 to 0.77) with excellent calibration (validation: calibration-in-the-large=0, slope=1.0). Patients with a score of at least 15 (n=4158, 19%) had a 62% mortality (positive predictive value 62%) compared with 1% mortality for those with a score of 3 or less (n=1650, 7%; negative predictive value 99%). Discriminatory performance was higher than 15 pre-existing risk stratification scores (area under the receiver operating characteristic curve range 0.61-0.76), with scores developed in other covid-19 cohorts often performing poorly (range 0.63-0.73).

Conclusions: An easy-to-use risk stratification score has been developed and validated based on commonly available parameters at hospital presentation. The 4C Mortality Score outperformed existing scores, showed utility to directly inform clinical decision making, and can be used to stratify patients admitted to hospital with covid-19 into different management groups. The score should be further validated to determine its applicability in other populations.

Study registration: ISRCTN66726260.

Fig 1

Model derivation and validation workflow. AUROC=area under the receiver operating characteristic curve; covid-19=coronavirus disease 2019; ISARIC CCP-UK=International Severe Acute Respiratory and emerging Infections Consortium Clinical Characterisation Protocol UK; NPV=negative predictive value; PPV=positive predictive value

Fig 2

Upper panel: distribution of patients across range of 4C Mortality Score in validation cohort; middle panel: observed in-hospital mortality across range of 4C Mortality Score in validation cohort; lower panel: predicted versus observed probability of in-hospital mortality (calibration; red line) for 4C Mortality Score within validation cohort

Fig 3

Receiver operator characteristic curves (upper panel) and decision curve analysis (lower panel) for most discriminating three models applicable to more than 50% of validation population compared with age alone (restricted cubic spline; imputed datasets). In lower panel, lines are shown for standardised net benefit at different risk thresholds of treating no patients (black line) and treating all patients (red dashed line)