Automated Calculation of Sequential Organ Failure Assessment (SOFA) Score in the Intensive Care Unit: Algorithm Development, Validation, and Association With 30-Day Mortality

Abstract

Background: Sequential Organ Failure Assessment (SOFA) score is routinely used in the intensive care unit (ICU) to describe severity of organ dysfunction, for prognostication and sepsis diagnosis, and in clinical trials. Inter-rater variability and scalability are known challenges in manual assessment. We aimed to develop and validate an algorithm for automatic SOFA calculation and evaluated its predictive abilities on 30-day mortality.

Methods: Retrospective multi-center cohort study on all adult patients admitted to four ICUs at the Karolinska University Hospitals in 2015-2018. Data from 2018 collected in one ICU was used for algorithm development. The algorithm was validated by comparing the results of automated SOFA score calculation to those obtained by manual SOFA scoring by experienced intensivists on 300 randomly chosen ICU days from the remaining cohort. Intra-class correlation coefficient (ICC [95% confidence interval (CI)]) was calculated as primary validation outcome. Area under the receiver operating characteristic curve (AUROC [95% CI]) was used for assessment of 30-day mortality prediction on the remaining cohort (excluding only the development cohort).

Results: A total of 6953 ICU admissions were included. The algorithm was developed on 613 admissions during 2018. Data from the remaining cohort with 6340 admissions (5076 patients, 36,625 ICU days) were used for mortality prediction. On algorithm validation of the full SOFA score, the ICC was 0.99 (0.98-1.00). For 30-day mortality, the best predictive abilities were found with maximum SOFA score (AUROC 0.79 [0.78-0.81]) and with SOFA score on Day 2 (AUROC 0.79 [0.77-0.80]).

Conclusion: A trustworthy automated SOFA score dataset can be produced with comprehensive high-frequency electronic health records curation and rigorous artifact control, with accuracy comparable to manual scoring by senior intensivists. Association between SOFA score and 30-day mortality in a large, real-world clinical cohort aligns with findings from previous clinical trials. The results support the use of automated SOFA scoring as a reliable tool for clinical research, quality monitoring, and potentially real-time clinical decision support.

Editorial comment: In this article, the authors report results of a retrospective multicenter study, where they used a large cohort of adult ICU patients for developing and validating an automated algorithm for calculating SOFA scores. In validation, they found that the automatically calculated score was comparable to scores manually calculated by experienced clinicians. The automatically calculated maximum SOFA and Day 2 SOFA scores performed well in predicting 30-day mortality.