Comparison of Preoperative Surgical Risk Estimated by Thoracic Surgeons vs a Standardized Surgical Risk Prediction Tool

Abstract

Considerable variability exists between surgeons' assessments of a patient's individual preoperative surgical risk. Surgical risk calculators are not routinely used despite their validation. We sought to compare thoracic surgeons' prediction of patients' risk of postoperative adverse outcomes vs a surgical risk calculator, the Surgical Risk Preoperative Assessment System (SURPAS). We developed vignettes from 30 randomly selected patients who underwent thoracic surgery in the American College of Surgeons' National Surgical Quality Improvement Program database. Twelve thoracic surgeons estimated patients' preoperative risks of postoperative morbidity and mortality. These were compared to SURPAS estimates of the same vignettes. C-indices and Brier scores were calculated for the surgeons' and SURPAS estimates. Agreement between surgeon estimates was examined using intraclass correlation coefficients (ICCs). Surgeons estimated higher morbidity risk compared to SURPAS for low-risk patients (ASA classes 1-2, 11.5% vs 5.1%, P ≤ 0.001) and lower morbidity risk compared to SURPAS for high-risk patients (ASA class 5, 37.6% vs 69.8%, P < 0.001). This trend also occurred in high-risk patients for mortality (ASA 5, 11.1% vs 44.3%, P < 0.001). C-indices for SURPAS vs surgeons were 0.84 vs 0.76 (P = 0.3) for morbidity and 0.98 vs 0.85 (P = 0.001) for mortality. Brier scores for SURPAS vs surgeons were 0.1579 vs 0.1986 for morbidity (P = 0.03) and 0.0409 vs 0.0543 for mortality (P = 0.006). ICCs showed that surgeons had moderate risk agreement for morbidity (ICC = 0.654) and mortality (ICC = 0.507). Thoracic surgeons and patients could benefit from using a surgical risk calculator to better estimate patients' surgical risks during the informed consent process.

Keywords: Outcomes; Risk; Risk assessment; SURPAS; Thoracic.