Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
, 269 (4), 652-662

MySurgeryRisk: Development and Validation of a Machine-learning Risk Algorithm for Major Complications and Death After Surgery

Affiliations

MySurgeryRisk: Development and Validation of a Machine-learning Risk Algorithm for Major Complications and Death After Surgery

Azra Bihorac et al. Ann Surg.

Abstract

Objective: To accurately calculate the risk for postoperative complications and death after surgery in the preoperative period using machine-learning modeling of clinical data.

Background: Postoperative complications cause a 2-fold increase in the 30-day mortality and cost, and are associated with long-term consequences. The ability to precisely forecast the risk for major complications before surgery is limited.

Methods: In a single-center cohort of 51,457 surgical patients undergoing major inpatient surgery, we have developed and validated an automated analytics framework for a preoperative risk algorithm (MySurgeryRisk) that uses existing clinical data in electronic health records to forecast patient-level probabilistic risk scores for 8 major postoperative complications (acute kidney injury, sepsis, venous thromboembolism, intensive care unit admission >48 hours, mechanical ventilation >48 hours, wound, neurologic, and cardiovascular complications) and death up to 24 months after surgery. We used the area under the receiver characteristic curve (AUC) and predictiveness curves to evaluate model performance.

Results: MySurgeryRisk calculates probabilistic risk scores for 8 postoperative complications with AUC values ranging between 0.82 and 0.94 [99% confidence intervals (CIs) 0.81-0.94]. The model predicts the risk for death at 1, 3, 6, 12, and 24 months with AUC values ranging between 0.77 and 0.83 (99% CI 0.76-0.85).

Conclusions: We constructed an automated predictive analytics framework for machine-learning algorithm with high discriminatory ability for assessing the risk of surgical complications and death using readily available preoperative electronic health records data. The feasibility of this novel algorithm implemented in real time clinical workflow requires further testing.

Conflict of interest statement

Conflict of Interest Disclosures: None reported.

Figures

Figure 1
Figure 1
Figure 1A. The conceptual framework of MySurgeryRisk analytics platform. The diagram shows sequence of steps from aggregation of raw data, data engineering and data analytics to final output. Figure 1B. The conceptual diagram of the Intelligent Perioperative Platform. This platform resides in a secure environment and in real time integrates and transforms electronic health records data, runs predictive algorithms, produces outputs for physicians, inputs their feedback and prospectively collects data for the future retraining of the prediction models
Figure 1
Figure 1
Figure 1A. The conceptual framework of MySurgeryRisk analytics platform. The diagram shows sequence of steps from aggregation of raw data, data engineering and data analytics to final output. Figure 1B. The conceptual diagram of the Intelligent Perioperative Platform. This platform resides in a secure environment and in real time integrates and transforms electronic health records data, runs predictive algorithms, produces outputs for physicians, inputs their feedback and prospectively collects data for the future retraining of the prediction models
Figure 2
Figure 2
MySurgeryRisk Output. The sample ouput for subjects with A, low mortality risk, and B, high mortality risk. Figure shows the predicted risks for eight postoperative complications for the given patient in eight equal-sized pies. The calculated cutoff values for AKI, ICU, MV, WND, CV, NEU, SEP, and VTE, were 0.35, 0.35, 0.13, 0.1, 0.07, 0.07, 0.06, and 0.03 respectively. Subjects are classified as high risk for a complication if calculated risk score exceeds the respective cutoff and respective pie is marked as red, and green otherwise. The size of the pie represents the proportion of the risk, scaled based on the cutoff for each complication. Green background color represents low mortality risk (Figure 2A) whereas red background color shows high mortality risk (Figure 2B). Abbreviation: AKI, acute kidney injury, CV, cardiovascular complications, ICU, intensive care unit addmission > 48 hours, MV, mechanical ventilation > 48 hours, NEU, neurologic complications, SEP, sepsis, VTE, venous thromboembolism, WND, wound complications.
Figure 3
Figure 3
Receiver operating characteristic curves and performance metrics for MySurgeryRisk algorithm in predicting A, more prevalent complications, B, less prevalent complications, and C, mortality.
Figure 3
Figure 3
Receiver operating characteristic curves and performance metrics for MySurgeryRisk algorithm in predicting A, more prevalent complications, B, less prevalent complications, and C, mortality.
Figure 3
Figure 3
Receiver operating characteristic curves and performance metrics for MySurgeryRisk algorithm in predicting A, more prevalent complications, B, less prevalent complications, and C, mortality.

Comment in

Similar articles

See all similar articles

Cited by 11 articles

See all "Cited by" articles

Publication types

Feedback