Visual Explanations From Deep 3D Convolutional Neural Networks for Alzheimer's Disease Classification

Abstract

We develop three efficient approaches for generating visual explanations from 3D convolutional neural networks (3D-CNNs) for Alzheimer's disease classification. One approach conducts sensitivity analysis on hierarchical 3D image segmentation, and the other two visualize network activations on a spatial map. Visual checks and a quantitative localization benchmark indicate that all approaches identify important brain parts for Alzheimer's disease diagnosis. Comparative analysis show that the sensitivity analysis based approach has difficulty handling loosely distributed cerebral cortex, and approaches based on visualization of activations are constrained by the resolution of the convo-lutional layer. The complementarity of these methods improves the understanding of 3D-CNNs in Alzheimer's disease classification from different perspectives.

Figure 1.

Left: The architecture of 3D-VGGNet; Middle: The architecture of 3D-ResNet; Right: The modified architecture of 3D-ResNet with global average pooling layer, 3D-ResNet-GAP, to produce 3D class activation mapping (3D-CAM). The only difference is that a global average pooling layer directly outputs to the softmax output layer (yellow boxes), replacing the original max pooling and fully connected layers.

Figure 2.

Horizontal, sagittal, and coronal view of the brain MRI and the visual explanation heatmaps.

Figure 3.

Precision-recall curve to localize cerebral cortex, lateral ventricle, and hippocampus regions using heatmaps.