Performance Evaluation of Prehospital Sepsis Prediction Models

Abstract

Objectives: Evaluate prediction models designed or used to identify patients with sepsis in the prehospital setting.

Design: Nested case-control study.

Setting: Four emergency departments (EDs) in Utah.

Patients: Adult nontrauma patient with available prehospital care records who received ED treatment during 2018 after arrival via ambulance.

Interventions: None.

Measurements and main results: Of 16,620 patients arriving to a study ED via ambulance, 1,037 (6.2%) met Sepsis-3 criteria in the ED. Complete prehospital care data was available for 434 case patients with sepsis and 434 control patients without sepsis. Model discrimination for the outcome of meeting Sepsis-3 criteria in the ED was quantified using the area under the precision-recall curve (AUPRC), which yields a value equal to outcome prevalence for a noninformative model. Of 21 evaluated prediction models, only the Prehospital Early Sepsis Detection (PRESEP) model (AUPRC, 0.33 [95% CI, 0.27-0.41) outperformed unaided infection assessment by emergency medical services (EMS) personnel (AUPRC, 0.17 [95% CI, 0.13-0.23]) for prehospital prediction of patients who would meet Sepsis-3 criteria in the ED ( p < 0.001). PRESEP also outperformed the quick Sequential Organ Failure Assessment score (AUPRC, 0.13 [95% CI, 0.11-0.16]; p < 0.001). Among 28 evaluated dichotomous predictors of ED sepsis, sensitivity ranged from 6% to 91% and positive predictive value 8-100%. PRESEP exhibited modest sensitivity (60%) and positive predictive value (20%).

Conclusions: PRESEP was the only evaluated prediction model that demonstrated better discrimination than unaided EMS infection assessment for the identification of ambulance-transported adult patients who met Sepsis-3 criteria in the ED.

Keywords: ambulance; emergency medical services; prediction model; screening; sepsis.