Objectives: Sepsis is a major public health concern with significant morbidity, mortality, and healthcare expenses. Early detection and antibiotic treatment of sepsis improve outcomes. However, although professional critical care societies have proposed new clinical criteria that aid sepsis recognition, the fundamental need for early detection and treatment remains unmet. In response, researchers have proposed algorithms for early sepsis detection, but directly comparing such methods has not been possible because of different patient cohorts, clinical variables and sepsis criteria, prediction tasks, evaluation metrics, and other differences. To address these issues, the PhysioNet/Computing in Cardiology Challenge 2019 facilitated the development of automated, open-source algorithms for the early detection of sepsis from clinical data.
Design: Participants submitted containerized algorithms to a cloud-based testing environment, where we graded entries for their binary classification performance using a novel clinical utility-based evaluation metric. We designed this scoring function specifically for the Challenge to reward algorithms for early predictions and penalize them for late or missed predictions and for false alarms.
Setting: ICUs in three separate hospital systems. We shared data from two systems publicly and sequestered data from all three systems for scoring.
Patients: We sourced over 60,000 ICU patients with up to 40 clinical variables for each hour of a patient's ICU stay. We applied Sepsis-3 clinical criteria for sepsis onset.
Measurements and main results: A total of 104 groups from academia and industry participated, contributing 853 submissions. Furthermore, 90 abstracts based on Challenge entries were accepted for presentation at Computing in Cardiology.
Conclusions: Diverse computational approaches predict the onset of sepsis several hours before clinical recognition, but generalizability to different hospital systems remains a challenge.
Conflict of interest statement
Drs. Reyna, Jeter, Nemati, and Clifford are partially funded by the National Science Foundation under award number 1822378 (Leveraging Heterogeneous Data Across International Borders in a Privacy Preserving Manner for Clinical Deep Learning). Mr. Shashikumar and Dr. Nemati are also funded by the National Institutes of Health (NIH) award number K01ES025445. Drs. Reyna, Josef, Jeter, Westover, Clifford, and Sharma received support for article research from the NIH. Dr. Josef’s institution received funding from NIH (T-32 Grant Trainee: T32GM095442-09) and Henry M. Jackson Foundation for role as a post-doctoral researcher for the Surgical Critical Care Institute,
Early Prediction of Sepsis From Clinical Data: The PhysioNet/Computing in Cardiology Challenge 2019.Crit Care Med. 2019 Dec 17. doi: 10.1097/CCM.0000000000004145. Online ahead of print. Crit Care Med. 2019. PMID: 31850926
An Interpretable Machine Learning Model for Accurate Prediction of Sepsis in the ICU.Crit Care Med. 2018 Apr;46(4):547-553. doi: 10.1097/CCM.0000000000002936. Crit Care Med. 2018. PMID: 29286945 Free PMC article. Clinical Trial.
The PhysioNet/Computing in Cardiology Challenge 2015: Reducing False Arrhythmia Alarms in the ICU.Comput Cardiol (2010). 2015 Sep;2015:273-276. doi: 10.1109/CIC.2015.7408639. Comput Cardiol (2010). 2015. PMID: 27331073 Free PMC article.
Systemic Inflammatory Response Syndrome.2019 Nov 21. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; 2020 Jan–. StatPearls. 2020 Jan–. PMID: 31613449 Free Books & Documents. Review.
An overview of mortality risk prediction in sepsis.Crit Care Med. 1995 Feb;23(2):376-93. doi: 10.1097/00003246-199502000-00026. Crit Care Med. 1995. PMID: 7867363 Review.
- Centers for Disease Control and Prevention: Sepsis. Cited 23 August 2016. https://www.cdc.gov/sepsis/datareports/index.html,. Accessed February 1 2019.
- World Health Organization: Sepsis. Cited 19 April 2018s. Available at: https://www.who.int/news-room/fact-sheets/detail/sepsis. Accessed February 1 2019.