End-to-end neural system identification with neural information flow

PLoS Comput Biol. 2021 Feb 4;17(2):e1008558. doi: 10.1371/journal.pcbi.1008558. eCollection 2021 Feb.


Neural information flow (NIF) provides a novel approach for system identification in neuroscience. It models the neural computations in multiple brain regions and can be trained end-to-end via stochastic gradient descent from noninvasive data. NIF models represent neural information processing via a network of coupled tensors, each encoding the representation of the sensory input contained in a brain region. The elements of these tensors can be interpreted as cortical columns whose activity encodes the presence of a specific feature in a spatiotemporal location. Each tensor is coupled to the measured data specific to a brain region via low-rank observation models that can be decomposed into the spatial, temporal and feature receptive fields of a localized neuronal population. Both these observation models and the convolutional weights defining the information processing within regions are learned end-to-end by predicting the neural signal during sensory stimulation. We trained a NIF model on the activity of early visual areas using a large-scale fMRI dataset recorded in a single participant. We show that we can recover plausible visual representations and population receptive fields that are consistent with empirical findings.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Adult
  • Brain / physiology*
  • Brain Mapping / methods*
  • Cognition
  • Humans
  • Learning
  • Magnetic Resonance Imaging / methods*
  • Male
  • Models, Neurological*
  • Neurons / physiology
  • Photic Stimulation
  • Semantics
  • Stochastic Processes
  • Television
  • Vision, Ocular
  • Visual Cortex / physiology*
  • Visual Perception / physiology*

Grant support

This research was supported by VIDI grant number 639.072.513 of The Netherlands Organization for Scientific Research (NWO, https://www.nwo.nl; (M. A. J. v. G.)). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.