Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
, 24, 102011

SVM Recursive Feature Elimination Analyses of Structural Brain MRI Predicts Near-Term Relapses in Patients With Clinically Isolated Syndromes Suggestive of Multiple Sclerosis


SVM Recursive Feature Elimination Analyses of Structural Brain MRI Predicts Near-Term Relapses in Patients With Clinically Isolated Syndromes Suggestive of Multiple Sclerosis

Viktor Wottschel et al. Neuroimage Clin.


Machine learning classification is an attractive approach to automatically differentiate patients from healthy subjects, and to predict future disease outcomes. A clinically isolated syndrome (CIS) is often the first presentation of multiple sclerosis (MS), but it is difficult at onset to predict who will have a second relapse and hence convert to clinically definite MS. In this study, we thus aimed to distinguish CIS converters from non-converters at onset of a CIS, using recursive feature elimination and weight averaging with support vector machines. We also sought to assess the influence of cohort size and cross-validation methods on the accuracy estimate of the classification. We retrospectively collected 400 patients with CIS from six European MAGNIMS MS centres. Patients underwent brain MRI at onset of a CIS according to local standard-of-care protocols. The diagnosis of clinically definite MS at one-year follow-up was the standard against which the accuracy of the model was tested. For each patient, we derived MRI-based features, such as grey matter probability, white matter lesion load, cortical thickness, and volume of specific cortical and white matter regions. Features with little contribution to the classification model were removed iteratively through an interleaved sample bootstrapping and feature averaging approach. Classification of CIS outcome at one-year follow-up was performed with 2-fold, 5-fold, 10-fold and leave-one-out cross-validation for each centre cohort independently and in all patients together. The estimated classification accuracy across centres ranged from 64.9% to 88.1% using 2-fold cross-validation and from 73% to 92.9% using leave-one-out cross-validation. The classification accuracy estimate was higher in single-centre, smaller data sets than in combinations of data sets, being the lowest when all patients were merged together. Regional MRI features such as WM lesions, grey matter probability in the thalamus and the precuneus or cortical thickness in the cuneus and inferior temporal gyrus predicted the occurrence of a second relapse in patients at onset of a CIS using support vector machines. The increased accuracy estimate of the classification achieved with smaller and single-centre samples may indicate a model bias (overfitting) when data points were limited, but also more homogeneous. We provide an overview of classifier performance from a range of cross-validation schemes to give insight into the variability across schemes. The proposed recursive feature elimination approach with weight averaging can be used both in single- and multi-centre data sets in order to bridge the gap between group-level comparisons and making predictions for individual patients.

Keywords: Feature selection; Machine learning classification; Multiple sclerosis.

Conflict of interest statement

The authors declare no potential conflicts of interest with respect to the research, authorship, or publication of this article.


Fig. 1:
Fig. 1
Illustration of the Neuromorphometrics atlas used for brain parcellation in this study.
Fig. 2:
Fig. 2
Accuracy estimates achieved at different iterations of the recursive feature selection when using all centres’ data sets combined together (BCGLMS)). The accuracy estimates increase with the first steps of the RFE, and the accuracy estimates generally increase with the number of folds. The shaded areas indicate 95% confidence intervals over 1000 bootstraps.
Fig. 3:
Fig. 3
Accuracy estimates per centre or combination of centres for each cross-validation method. Corresponding values for confidence intervals, sensitivity and specificity can be found in Tables 2–5.
Fig. 4:
Fig. 4
Top: bar chart of the proportion of converters in the cohort. Bottom: estimated classification accuracy relative to the size of the minority class. There is a general increase of estimated accuracy with a decrease in sample size. The subscript M and S indicate multi-centre and single-centre data sets respectively.
Fig. 5:
Fig. 5
Location of features relevant to the prediction of CIS conversion at 1-year follow-up. The highlighted areas represent A: GM probability, B: regional volume sizes and C: cortical thickness respectively. Please note that white matter lesion load across the whole brain was also selected but is not shown here for clarity. Type of CIS onset was selected as the only non-imaging feature. A full list of features can be found in the supplementary material.

Similar articles

See all similar articles


    1. Miller DH, Chard DT, Ciccarelli O. Clinically isolated syndromes. Lancet Neurol. 2012;11(2):157–169. - PubMed
    1. Tintore M, À Rovira, Río J. Defining high, medium and low impact prognostic factors for developing multiple sclerosis. Brain. 2015;138(7):1863–1874. - PubMed
    1. Weygandt M, Hackmack K, Pfüller C. MRI pattern recognition in multiple sclerosis normal-appearing brain areas. PLoS One. 2011;6(6):e21138. Kleinschnitz C, ed. - PMC - PubMed
    1. Bendfeldt K, Klöppel S, Nichols TE. Multivariate pattern classification of gray matter pathology in multiple sclerosis. Neuroimage. 2012;60(1):400–408. - PubMed
    1. Wottschel V, Alexander DC, Kwok PP. Predicting outcome in clinically isolated syndrome using machine learning. NeuroImage Clin. 2015;7:281–287. - PMC - PubMed

LinkOut - more resources