Sparse deep predictive coding captures contour integration capabilities of the early visual system

PLoS Comput Biol. 2021 Jan 26;17(1):e1008629. doi: 10.1371/journal.pcbi.1008629. eCollection 2021 Jan.


Both neurophysiological and psychophysical experiments have pointed out the crucial role of recurrent and feedback connections to process context-dependent information in the early visual cortex. While numerous models have accounted for feedback effects at either neural or representational level, none of them were able to bind those two levels of analysis. Is it possible to describe feedback effects at both levels using the same model? We answer this question by combining Predictive Coding (PC) and Sparse Coding (SC) into a hierarchical and convolutional framework applied to realistic problems. In the Sparse Deep Predictive Coding (SDPC) model, the SC component models the internal recurrent processing within each layer, and the PC component describes the interactions between layers using feedforward and feedback connections. Here, we train a 2-layered SDPC on two different databases of images, and we interpret it as a model of the early visual system (V1 & V2). We first demonstrate that once the training has converged, SDPC exhibits oriented and localized receptive fields in V1 and more complex features in V2. Second, we analyze the effects of feedback on the neural organization beyond the classical receptive field of V1 neurons using interaction maps. These maps are similar to association fields and reflect the Gestalt principle of good continuation. We demonstrate that feedback signals reorganize interaction maps and modulate neural activity to promote contour integration. Third, we demonstrate at the representational level that the SDPC feedback connections are able to overcome noise in input images. Therefore, the SDPC captures the association field principle at the neural level which results in a better reconstruction of blurred images at the representational level.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Algorithms
  • Animals
  • Computational Biology
  • Deep Learning*
  • Feedback
  • Female
  • Humans
  • Image Processing, Computer-Assisted
  • Male
  • Models, Neurological*
  • Visual Cortex / physiology
  • Visual Pathways*

Grant support

VB and LP received funding from the European Union’s H2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement n 713750, by the Regional Council of Provence-Alpes-Côte d’Azur, A*MIDEX (n ANR-11-IDEX-0001-02), and the financial support of ANR project “Horizontal-V1” (n ANR-17-CE37-0006). This work was granted access to the HPC resources of Aix-Marseille Université financed by the project Equip@Meso (n ANR-10-EQPX-29-01) of the program "Investissements d’Avenir". Other authors received no specific funding for this work. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.