Machine learning versus regression for prediction of sporadic pancreatic cancer

Abstract

Background/objectives: There is currently no widely accepted approach to identify patients at increased risk for sporadic pancreatic cancer (PC). We aimed to compare the performance of two machine-learning models with a regression-based model in predicting pancreatic ductal adenocarcinoma (PDAC), the most common form of PC.

Methods: This retrospective cohort study consisted of patients 50-84 years of age enrolled in either Kaiser Permanente Southern California (KPSC, model training, internal validation) or the Veterans Affairs (VA, external testing) between 2008 and 2017. The performance of random survival forests (RSF) and eXtreme gradient boosting (XGB) models were compared to that of COX proportional hazards regression (COX). Heterogeneity of the three models were assessed.

Results: The KPSC and the VA cohorts consisted of 1.8 and 2.7 million patients with 1792 and 4582 incident PDAC cases within 18 months, respectively. Predictors selected into all three models included age, abdominal pain, weight change, and glycated hemoglobin (A1c). Additionally, RSF selected change in alanine transaminase (ALT), whereas the XGB and COX selected the rate of change in ALT. The COX model appeared to have lower AUC (KPSC: 0.737, 95% CI 0.710-0.764; VA: 0.706, 0.699-0.714), compared to those of RSF (KPSC: 0.767, 0.744-0.791; VA: 0.731, 0.724-0.739) and XGB (KPSC: 0.779, 0.755-0.802; VA: 0.742, 0.735-0.750). Among patients with top 5% predicted risk from all three models (N = 29,663), 117 developed PDAC, of which RSF, XGB and COX captured 84 (9 unique), 87 (4 unique), 87 (19 unique) cases, respectively.

Conclusions: The three models complement each other, but each has unique contributions.

Keywords: Machine learning versus regression; Pancreatic cancer; Random survival forest; Risk prediction; eXtreme gradient boosting.