Functional Connectivity-Based Prediction of Autism on Site Harmonized ABIDE Dataset

Abstract

Objective: The larger sample sizes available from multi-site publicly available neuroimaging data repositories makes machine-learning based diagnostic classification of mental disorders more feasible by alleviating the curse of dimensionality. However, since multi-site data are aggregated post-hoc, i.e. they were acquired from different scanners with different acquisition parameters, non-neural inter-site variability may mask inter-group differences that are at least in part neural in origin. Hence, the advantages gained by the larger sample size in the context of machine-learning based diagnostic classification may not be realized.

Methods: We address this issue using harmonization of multi-site neuroimaging data using the ComBat technique, which is based on an empirical Bayes formulation to remove inter-site differences in data distributions, to improve diagnostic classification accuracy. Specifically, we demonstrate this using ABIDE (Autism Brain Imaging Data Exchange) multi-site data for classifying individuals with Autism from healthy controls using resting state fMRI-based functional connectivity data.

Results: Our results show that higher classification accuracies across multiple classification models can be obtained (especially for models based on artificial neural networks) from multi-site data post harmonization with the ComBat technique as compared to without harmonization, outperforming earlier results from existing studies using ABIDE. Furthermore, our network ablation analysis facilitated important insights into autism spectrum disorder pathology and the connectivity in networks shown to be important for classification covaried with verbal communication impairments in Autism.

Conclusion: Multi-site data harmonization using ComBat improves neuroimaging-based diagnostic classification of mental disorders.

Significance: ComBat has the potential to make AI-based clinical decision-support systems more feasible in psychiatry.

Fig. 1.

A schematic diagram of all the classification methods used. An artificial neural network (ANN) based classifier was implemented along with a Random forest (RF) of classification trees. Architecture for classification involving denoising autoencoders based on Heinsfeld et. al. has been shown.

Fig. 2.

Bar chart showing the site-specific accuracy, sensitivity and specificity obtained from harmonized as well as non-harmonized data for the three methods (random forests, artificial neural networks and Heinsfeld’s auto-encoders) employed.

Fig. 3.

Comparison of Area under receiver-operating characteristic (AU-ROC) between harmonized and non-harmonized datasets for all the classification methods used.

Fig. 4.

Brain maps showing ROIs associated with each of the 12 sub-networks used in the ablation analysis. Table S1 in the supplement provides further information about each sub-network, such as ROIs in each sub-network, their names and MNI centroids.

Fig. 5.

Results from the ablation analysis of harmonized data with the ANN classifier. The drop in accuracy due to occluding every sub-network can be observed per test site (LOSO). Positive values indicate a drop in accuracy due to ablation.

Fig. 6.

The percentage drop in accuracy (the median and range is shown) across all sites when each of the sub-networks are occluded in the ablation analysis (top). The frequency of drop in accuracy, i.e. the number of sites where in a drop in accuracy is observed, for occlusion of each of the sub-networks in ablation analysis (bottom).

Fig. 7.

Correlation between characteristic path length obtained from the FC matrices of the auditory network in Autism subjects with the ADIR verbal scores in those subjects.