Multistep validation of a post-ERCP pancreatitis prediction system integrating multimodal data: a multicenter study

Abstract

Background and aims: The impact of various categories of information on the prediction of post-ERCP pancreatitis (PEP) remains uncertain. We comprehensively investigated the risk factors associated with PEP by constructing and validating a model incorporating multimodal data through multiple steps.

Methods: Cases (n = 1916) of ERCP were retrospectively collected from multiple centers for model construction. Through literature research, 49 electronic health record (EHR) features and 1 image feature related to PEP were identified. The EHR features were categorized into baseline, diagnosis, technique, and prevention strategies, covering pre-ERCP, intra-ERCP, and peri-ERCP phases. We first incrementally constructed models 1 to 4 incorporating these 4 feature categories and then added the image feature into models 1 to 4 and developed models 5 to 8. All models underwent testing and comparison using both internal and external test sets. Once the optimal model was selected, we conducted comparisons among multiple machine learning algorithms.

Results: Compared with model 2 that incorporated baseline and diagnosis features, adding technique and prevention strategies (model 4) greatly improved the sensitivity (63.89% vs 83.33%, P < .05) and specificity (75.00% vs 85.92%, P < .001). A similar tendency was observed in the internal and external tests. In model 4, the top 3 features ranked by weight were previous pancreatitis, nonsteroidal anti-inflammatory drug use, and difficult cannulation. The image-based feature has the highest weight in models 5 to 8. Finally, model 8 used a random forest algorithm and showed the best performance.

Conclusions: We first developed a multimodal prediction model for identifying PEP with a clinical-acceptable performance. The image and technique features are crucial for PEP prediction.