Development of supervised machine learning algorithms for prediction of satisfaction at 2 years following total shoulder arthroplasty

Abstract

Background: Patient satisfaction after primary anatomic and reverse total shoulder arthroplasty (TSA) represents an important metric for gauging patients' perception of their care and surgical outcomes. Although TSA confers improvement in pain and function for most patients, inevitably some will remain unsatisfied postoperatively. The purpose of this study was to (1) train supervised machine learning (SML) algorithms to predict satisfaction after TSA and (2) develop a clinical tool for individualized assessment of patient-specific risk factors.

Methods: We performed a retrospective review of primary anatomic and reverse TSA patients between January 2014 and February 2018. A total of 16 demographic, clinical, and patient-reported outcomes were evaluated for predictive value. Five SML algorithms underwent 3 iterations of 10-fold cross-validation on a training set (80% of cohort). Assessment by discrimination, calibration, Brier score, and decision-curve analysis was performed on an independent testing set (remaining 20% of cohort). Global and local model behaviors were evaluated with global variable importance plots and local interpretable model-agnostic explanations, respectively.

Results: The study cohort consisted of 413 patients, of whom 331 (82.6%) were satisfied at 2 years postoperatively. The support vector machine model demonstrated the best relative performance on the independent testing set not used for model training (concordance statistic, 0.80; calibration intercept, 0.20; calibration slope, 2.32; Brier score, 0.11). The most important factors for predicting satisfaction were baseline Single Assessment Numeric Evaluation score, exercise and activity, workers' compensation status, diagnosis, symptom duration prior to surgery, body mass index, age, smoking status, anatomic vs. reverse TSA, and diabetes. The support vector machine algorithm was incorporated into an open-access digital application for patient-level explanations of risk and predictions, available at https://orthopedics.shinyapps.io/SatisfactionTSA/.

Conclusion: The best-performing SML model demonstrated excellent discrimination and adequate calibration for predicting satisfaction following TSA and was used to create an open-access, clinical decision-making tool. However, rigorous external validation in different geographic locations and patient populations is essential prior to assessment of clinical utility. Given that this tool is based on partially modifiable risk factors, it may enhance shared decision making and allow for periods of targeted preoperative health-optimization efforts.

Keywords: Total shoulder arthroplasty; classification; cross-validation; feature selection; satisfaction; supervised machine learning (SML); support vector machine (SVM).