Comparison of prospective risk estimates for postoperative complications: human vs computer model

Abstract

Background: Surgical quality improvement tools such as NSQIP are limited in their ability to prospectively affect individual patient care by the retrospective audit and feedback nature of their design. We hypothesized that statistical models using patient preoperative characteristics could prospectively provide risk estimates of postoperative adverse events comparable to risk estimates provided by experienced surgeons, and could be useful for stratifying preoperative assessment of patient risk.

Study design: This was a prospective observational cohort. Using previously developed models for 30-day postoperative mortality, overall morbidity, cardiac, thromboembolic, pulmonary, renal, and surgical site infection (SSI) complications, model and surgeon estimates of risk were compared with each other and with actual 30-day outcomes.

Results: The study cohort included 1,791 general surgery patients operated on between June 2010 and January 2012. Observed outcomes were mortality (0.2%), overall morbidity (8.2%), and pulmonary (1.3%), cardiac (0.3%), thromboembolism (0.2%), renal (0.4%), and SSI (3.8%) complications. Model and surgeon risk estimates showed significant correlation (p < 0.0001) for each outcome category. When surgeons perceived patient risk for overall morbidity to be low, the model-predicted risk and observed morbidity rates were 2.8% and 4.1%, respectively, compared with 10% and 18% in perceived high risk patients. Patients in the highest quartile of model-predicted risk accounted for 75% of observed mortality and 52% of morbidity.

Conclusions: Across a broad range of general surgical operations, we confirmed that the model risk estimates are in fairly good agreement with risk estimates of experienced surgeons. Using these models prospectively can identify patients at high risk for morbidity and mortality, who could then be targeted for intervention to reduce postoperative complications.

Keywords: ACS; ASA; American College of Surgeons; American Society of Anesthesiologists; BMI; DS3; Decision Support for Safer Surgery; RVU; SSI; body mass index; relative value units; surgical site infection.

Figure 1

(A) Morbidity, mortality and complication occurrences by model prediction of risk. (B) Morbidity, mortality and complication occurrences by attending prediction of risk. DVT, deep vein thrombosis; Pred, predicted; SSI, surgical site infection.

Figure 1

(A) Morbidity, mortality and complication occurrences by model prediction of risk. (B) Morbidity, mortality and complication occurrences by attending prediction of risk. DVT, deep vein thrombosis; Pred, predicted; SSI, surgical site infection.