Visual and Haptic Shape Processing in the Human Brain: Unisensory Processing, Multisensory Convergence, and Top-Down Influences

Cereb Cortex. 2016 Aug;26(8):3402-3412. doi: 10.1093/cercor/bhv170. Epub 2015 Jul 28.


Humans are highly adept at multisensory processing of object shape in both vision and touch. Previous studies have mostly focused on where visually perceived object-shape information can be decoded, with haptic shape processing receiving less attention. Here, we investigate visuo-haptic shape processing in the human brain using multivoxel correlation analyses. Importantly, we use tangible, parametrically defined novel objects as stimuli. Two groups of participants first performed either a visual or haptic similarity-judgment task. The resulting perceptual object-shape spaces were highly similar and matched the physical parameter space. In a subsequent fMRI experiment, objects were first compared within the learned modality and then in the other modality in a one-back task. When correlating neural similarity spaces with perceptual spaces, visually perceived shape was decoded well in the occipital lobe along with the ventral pathway, whereas haptically perceived shape information was mainly found in the parietal lobe, including frontal cortex. Interestingly, ventrolateral occipito-temporal cortex decoded shape in both modalities, highlighting this as an area capable of detailed visuo-haptic shape processing. Finally, we found haptic shape representations in early visual cortex (in the absence of visual input), when participants switched from visual to haptic exploration, suggesting top-down involvement of visual imagery on haptic shape processing.

Keywords: haptics; multisensory; multivoxel analysis; shape perception.

MeSH terms

  • Adolescent
  • Adult
  • Brain / diagnostic imaging
  • Brain / physiology*
  • Brain Mapping
  • Female
  • Humans
  • Judgment / physiology
  • Learning / physiology
  • Magnetic Resonance Imaging
  • Male
  • Neural Pathways / diagnostic imaging
  • Neural Pathways / physiology
  • Neuropsychological Tests
  • Random Allocation
  • Touch Perception / physiology*
  • Visual Perception / physiology*
  • Young Adult