MOIS-SAM2: Exemplar-based segment anything model 2 for multi-lesion interactive segmentation of neurofibromas in whole-body MRI

Abstract

Background and objectives: Neurofibromatosis type 1 is a genetic disorder characterized by the development of numerous neurofibromas (NFs) throughout the body. Whole-body MRI (WB-MRI) is the clinical standard for detection and longitudinal surveillance of NF tumor growth; however, manual segmentation of these lesions is labor-intensive. Existing interactive segmentation methods fail to combine high lesion-wise precision with scalability to hundreds of lesions. This study proposes a novel interactive segmentation model tailored to address this challenge.

Methods: We introduce MOIS-SAM2 - a multi-object interactive segmentation model that extends the state-of-the-art, transformer-based, promptable Segment Anything Model 2 (SAM2) with exemplar-based semantic propagation. The model implements user prompts to segment a small set of lesions and propagates this knowledge to similar, unprompted lesions across the entire scan. In this retrospective study, MOIS-SAM2 was trained and evaluated on 119 WB-MRI scans from 84 NF1 patients acquired using T2-weighted fat-suppressed sequences. The dataset was split at the patient level into a training set and four test sets (one in-domain and three reflecting different domain shift scenarios, e.g., MRI field strength variation, low tumor burden, differences in clinical site and scanner vendor). Segmentation performance was assessed using scan-wise Dice Similarity Coefficient (DSC), lesion detection F1 score, and lesion-wise DSC.

Results: On the in-domain test set, MOIS-SAM2 achieved a scan-wise DSC of 0.60 against expert manual annotations, outperforming the baseline 3D nnU-Net (DSC: 0.54) and SAM2 (DSC: 0.35). The performance of the proposed model was maintained under MRI field strength shift (DSC: 0.53) and scanner vendor variation (DSC: 0.50), and improved in low tumor burden cases (DSC: 0.61). Lesion detection F1 scores ranged from 0.62 to 0.78 across test sets. Preliminary inter-reader variability analysis showed model-to-expert agreement (DSC: 0.62-0.68), comparable to inter-expert agreement (DSC: 0.57-0.69).

Conclusions: The proposed MOIS-SAM2 enables efficient and scalable interactive segmentation of NFs in WB-MRI with minimal user input and strong generalization, supporting integration into clinical workflows. The model and code are publicly available on GitHub.

Keywords: Exemplar learning; Interactive image segmentation; Medical image analysis; Multi-lesion segmentation; Neurofibroma; Segment anything model 2; Whole-body MRI.