Temporal embedding and spatiotemporal feature selection boost multi-voxel pattern analysis decoding accuracy

Jeiran Choupan; Pamela K Douglas; Yaniv Gal; Mark S Cohen; David C Reutens; Zhengyi Yang

doi:10.1016/j.jneumeth.2020.108836

Temporal embedding and spatiotemporal feature selection boost multi-voxel pattern analysis decoding accuracy

J Neurosci Methods. 2020 Nov 1:345:108836. doi: 10.1016/j.jneumeth.2020.108836. Epub 2020 Jul 26.

Authors

Jeiran Choupan¹, Pamela K Douglas², Yaniv Gal³, Mark S Cohen⁴, David C Reutens⁵, Zhengyi Yang⁶

Affiliations

¹ Centre for Advanced Imaging, The University of Queensland, Brisbane, Australia; Queensland Brain Institute, The University of Queensland, Brisbane, Australia; Department of Psychology, USC Dornsife College of Letters, Arts and Sciences, University of Southern California, USA; Laboratory of Neuro Imaging, USC Stevens Neuroimaging and Informatics Institute, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA. Electronic address: choupan@usc.edu.
² Center for Cognitive Neuroscience, University of California, Los Angeles, CA, USA; Modeling & Simulation, and Computer Science Departments, UCF, Florida, USA.
³ School of Information Technology and Electrical Engineering, The University of Queensland, Brisbane, Australia.
⁴ Neuropsychiatric Institute, University of California, Los Angeles, CA, USA; Departments of Psychiatry and Behavioral Sciences, Neurology, Radiological Sciences, Biomedical Physics, Psychology, and Bioengineering, University of California, Los Angeles, CA, USA; California Nanosystems Institute UCLA School of Medicine, Los Angeles, CA, USA.
⁵ Centre for Advanced Imaging, The University of Queensland, Brisbane, Australia.
⁶ Centre for Advanced Imaging, The University of Queensland, Brisbane, Australia; School of Information Technology and Electrical Engineering, The University of Queensland, Brisbane, Australia; Brainnetome Center, Institute of Automation, Chinese Academy of Sciences, Beijing, China.

Abstract

Background: In fMRI decoding, temporal embedding of spatial features of the brain allows the incorporation of brain activity dynamics into the multivariate pattern classification process, and provides enriched information about stimulus-specific response patterns and potentially improved prediction accuracy.

New method: This study investigates the possibility of enhancing the classification performance by exploring temporal embedding, to identify the optimum combination of spatiotemporal features based on their classification performance. We investigated the importance of spatiotemporal feature selection using a slow event-related design adapted from the classic Haxby study (Haxby et al., 2001). Data were collected using a multiband fMRI sequence with temporal resolution of 0.568 s.

Comparison with existing methods: A wide range of spatiotemporal observations were created as various combinations of spatiotemporal features. Using both random forest, and support vector machine, classifiers prediction accuracies for these combinations were then compared with the single spatial multivariate pattern approach that uses only a single temporal observation.

Results: Our findings showed that, on average, spatiotemporal feature selection improved prediction accuracy. Moreover, the random forest algorithm outperformed the support vector machine and benefitted from temporal information to a greater extent.

Conclusions: As expected, the most influential temporal durations were found to be around the peak of the hemodynamic response function, a few seconds after the stimuli onset until -4 s after the peak of the hemodynamic response function. The superiority of spatiotemporal feature selection over single time-point spatial approaches invites future work to design optimal approaches that incorporate spatiotemporal dependencies into feature selection for decoding.

Keywords: Multi-variate pattern analysis; Multiband EPI; Random forest; Spatiotemporal feature selection; Support vector machine; fMRI.

Publication types

Research Support, N.I.H., Extramural
Research Support, Non-U.S. Gov't

MeSH terms

Algorithms*
Brain Mapping
Magnetic Resonance Imaging
Pattern Recognition, Automated*
Support Vector Machine

Abstract

Publication types

MeSH terms

Grants and funding