Prediction of Critical Care Outcome for Adult Patients Presenting to Emergency Department Using Initial Triage Information: An XGBoost Algorithm Analysis

Hyoungju Yun; Jinwook Choi; Jeong Ho Park

doi:10.2196/30770

Prediction of Critical Care Outcome for Adult Patients Presenting to Emergency Department Using Initial Triage Information: An XGBoost Algorithm Analysis

JMIR Med Inform. 2021 Sep 20;9(9):e30770. doi: 10.2196/30770.

Authors

Hyoungju Yun¹, Jinwook Choi^{1

2

3}, Jeong Ho Park^{4

5}

Affiliations

¹ Interdisciplinary Program of Medical Informatics, College of Medicine, Seoul National University, Seoul, Republic of Korea.
² Department of Biomedical Engineering, College of Medicine, Seoul National University, Seoul, Republic of Korea.
³ Institute of Medical and Biological Engineering, Medical Research Center, Seoul National University, Seoul, Republic of Korea.
⁴ Department of Emergency Medicine, College of Medicine, Seoul National University, Seoul, Republic of Korea.
⁵ Laboratory of Emergency Medical Services, Biomedical Research Institute, Seoul National University Hospital, Seoul, Republic of Korea.

PMID: 34346889
PMCID: PMC8491120
DOI: 10.2196/30770

Abstract

Background: The emergency department (ED) triage system to classify and prioritize patients from high risk to less urgent continues to be a challenge.

Objective: This study, comprising 80,433 patients, aims to develop a machine learning algorithm prediction model of critical care outcomes for adult patients using information collected during ED triage and compare the performance with that of the baseline model using the Korean Triage and Acuity Scale (KTAS).

Methods: To predict the need for critical care, we used 13 predictors from triage information: age, gender, mode of ED arrival, the time interval between onset and ED arrival, reason of ED visit, chief complaints, systolic blood pressure, diastolic blood pressure, pulse rate, respiratory rate, body temperature, oxygen saturation, and level of consciousness. The baseline model with KTAS was developed using logistic regression, and the machine learning model with 13 variables was generated using extreme gradient boosting (XGB) and deep neural network (DNN) algorithms. The discrimination was measured by the area under the receiver operating characteristic (AUROC) curve. The ability of calibration with Hosmer-Lemeshow test and reclassification with net reclassification index were evaluated. The calibration plot and partial dependence plot were used in the analysis.

Results: The AUROC of the model with the full set of variables (0.833-0.861) was better than that of the baseline model (0.796). The XGB model of AUROC 0.861 (95% CI 0.848-0.874) showed a higher discriminative performance than the DNN model of 0.833 (95% CI 0.819-0.848). The XGB and DNN models proved better reclassification than the baseline model with a positive net reclassification index. The XGB models were well-calibrated (Hosmer-Lemeshow test; P>.05); however, the DNN showed poor calibration power (Hosmer-Lemeshow test; P<.001). We further interpreted the nonlinear association between variables and critical care prediction.

Conclusions: Our study demonstrated that the performance of the XGB model using initial information at ED triage for predicting patients in need of critical care outperformed the conventional model with KTAS.

Keywords: XGBoost; algorithm; classify; critical care; emergency; explainable machine learning; interpretable artificial intelligence; machine learning; model; outcome; prediction; prioritize; risk; triage.

©Hyoungju Yun, Jinwook Choi, Jeong Ho Park. Originally published in JMIR Medical Informatics (https://medinform.jmir.org), 20.09.2021.