Development and validation of a machine learning model to predict postoperative delirium using a nationwide database: A retrospective, observational study

Abstract

Study objective: Postoperative delirium is a neuropsychological syndrome that typically occurs in surgical patients. Its onset can lead to prolonged hospitalization as well as increased morbidity and mortality. Therefore, it is important to promptly identify its signs. This study aimed to develop and validate a machine learning predictive model for postoperative delirium using extensive population data.

Design: Retrospective observational study.

Setting: Japanese Diagnosis Procedure Combination inpatient data. Data were used for internal (2016.4-2018.12) and temporal validation (2019.01-2019.10).

Patients: Patients aged ≥65 years who underwent general anesthesia for surgical procedure.

Measurements: The primary outcome was postoperative delirium, which was defined as a condition requiring newly prescribed antipsychotic drugs or assignment of the corresponding insurance claim code after the date of surgery. We trained and tuned the optimal machine-learning model through 10-fold cross-validation using the selected optimal area under the receiver operating characteristic curve (AUC) value. In the temporal validation, we measured the performance of our model.

Main results: The analysis included 557,990 patients. The light-gradient boosting machine models showed a higher AUC value (0.826 [95% confidence interval (CI): 0.822-0.829]) than the other models. Regarding performance, the model had a recall value of 0.124 (95% CI: 0.119-0.129) and precision value of 0.659 (95% CI: 0.641-0.677]). This performance was sustained in the temporal validation (AUC, 0.815 [95% CI: 0.811-0.818]). At a sensitivity of 0.80, the model achieved a specificity of 0.672 (95% CI: 0.670-0.674]), a negative predictive value of 0.975 (95% CI: 0.974-0.975), and a positive predictive value of 0.176 (95% CI: 0.176-0.179).

Conclusions: Using extensive Diagnostic Procedure Combination data, we successfully created and validated a machine learning model for predicting postoperative delirium. This model may facilitate prediction of postoperative delirium.

Keywords: Machine learning model; Postoperative delirium; Predictive model.