Brain hierarchy score: Which deep neural networks are hierarchically brain-like?

Soma Nonaka; Kei Majima; Shuntaro C Aoki; Yukiyasu Kamitani

doi:10.1016/j.isci.2021.103013

Brain hierarchy score: Which deep neural networks are hierarchically brain-like?

iScience. 2021 Aug 21;24(9):103013. doi: 10.1016/j.isci.2021.103013. eCollection 2021 Sep 24.

Authors

Soma Nonaka¹, Kei Majima², Shuntaro C Aoki², Yukiyasu Kamitani^{2

3}

Affiliations

¹ Faculty of Integrated Human Studies, Kyoto University, Kyoto 606-8501, Japan.
² Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan.
³ ATR Computational Neuroscience Laboratories, Seika, Kyoto 619-0288, Japan.

Abstract

Achievement of human-level image recognition by deep neural networks (DNNs) has spurred interest in whether and how DNNs are brain-like. Both DNNs and the visual cortex perform hierarchical processing, and correspondence has been shown between hierarchical visual areas and DNN layers in representing visual features. Here, we propose the brain hierarchy (BH) score as a metric to quantify the degree of hierarchical correspondence based on neural decoding and encoding analyses where DNN unit activations and human brain activity are predicted from each other. We find that BH scores for 29 pre-trained DNNs with various architectures are negatively correlated with image recognition performance, thus indicating that recently developed high-performance DNNs are not necessarily brain-like. Experimental manipulations of DNN models suggest that single-path sequential feedforward architecture with broad spatial integration is critical to brain-like hierarchy. Our method may provide new ways to design DNNs in light of their representational homology to the brain.

Keywords: Human-centered computing; Neural networks; Neuroscience.

Associated data

figshare/10.6084/m9.figshare.12401168