Multimodal Evidence on Shape and Surface Information in Individual Face Processing

Neuroimage. 2019 Jan 1;184:813-825. doi: 10.1016/j.neuroimage.2018.09.083. Epub 2018 Oct 3.

Abstract

The significance of shape and surface information for face perception is well established, yet their relative contribution to recognition and their neural underpinnings await clarification. Here, we employ image reconstruction to retrieve, assess and visualize such information using behavioral, electroencephalography and functional magnetic resonance imaging data. Our results indicate that both shape and surface information can be successfully recovered from each modality but that the latter is better recovered than the former, consistent with its key role for face representations. Further, shape and surface information exhibit similar spatiotemporal profiles, rely on the extraction of specific visual features, such as eye shape or skin tone, and reveal a systematic representational structure, albeit with more cross-modal consistency for shape than surface. More generally, the present work illustrates a novel approach to relating and comparing different modalities in terms of perceptual information content. Thus, our results help elucidate the representational basis of individual face recognition while, methodologically, they showcase the utility of image reconstruction and clarify its reliance on diagnostic visual information.

Keywords: EEG; Face recognition; Image reconstruction; fMRI.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Adolescent
  • Adult
  • Brain Mapping / methods
  • Electroencephalography
  • Facial Recognition / physiology*
  • Female
  • Form Perception / physiology*
  • Humans
  • Image Processing, Computer-Assisted / methods*
  • Magnetic Resonance Imaging
  • Male
  • Photic Stimulation
  • Young Adult