Object segmentation controls image reconstruction from natural scenes

PLoS Biol. 2017 Aug 21;15(8):e1002611. doi: 10.1371/journal.pbio.1002611. eCollection 2017 Aug.


The structure of the physical world projects images onto our eyes. However, those images are often poorly representative of environmental structure: well-defined boundaries within the eye may correspond to irrelevant features of the physical world, while critical features of the physical world may be nearly invisible at the retinal projection. The challenge for the visual cortex is to sort these two types of features according to their utility in ultimately reconstructing percepts and interpreting the constituents of the scene. We describe a novel paradigm that enabled us to selectively evaluate the relative role played by these two feature classes in signal reconstruction from corrupted images. Our measurements demonstrate that this process is quickly dominated by the inferred structure of the environment, and only minimally controlled by variations of raw image content. The inferential mechanism is spatially global and its impact on early visual cortex is fast. Furthermore, it retunes local visual processing for more efficient feature extraction without altering the intrinsic transduction noise. The basic properties of this process can be partially captured by a combination of small-scale circuit models and large-scale network architectures. Taken together, our results challenge compartmentalized notions of bottom-up/top-down perception and suggest instead that these two modes are best viewed as an integrated perceptual mechanism.

Publication types

  • Comparative Study

MeSH terms

  • Algorithms
  • Biomarkers / analysis
  • Brain Mapping
  • Electroencephalography
  • Female
  • Humans
  • Male
  • Models, Neurological*
  • Nerve Net / physiology
  • Neurons / physiology*
  • Retina / physiology*
  • Retinal Neurons / physiology*
  • Spatial Processing
  • Vision, Ocular*
  • Visual Cortex / physiology*
  • Visual Fields
  • Visual Pathways / physiology
  • Visual Perception*


  • Biomarkers

Grant support

This work was supported by a University Research Fellowship from the Royal Society of London (UK); the CNRS (France); grants ANR-10-LABX-0087 IEC, ANR-10-IDEX-0001-02 PSL and ANR-16-CE28-0016 from the Agence Nationale de la Recherche. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.