Integration of auditory and visual information about objects in superior temporal sulcus

Neuron. 2004 Mar 4;41(5):809-23. doi: 10.1016/s0896-6273(04)00070-4.


Two categories of objects in the environment-animals and man-made manipulable objects (tools)-are easily recognized by either their auditory or visual features. Although these features differ across modalities, the brain integrates them into a coherent percept. In three separate fMRI experiments, posterior superior temporal sulcus and middle temporal gyrus (pSTS/MTG) fulfilled objective criteria for an integration site. pSTS/MTG showed signal increases in response to either auditory or visual stimuli and responded more to auditory or visual objects than to meaningless (but complex) control stimuli. pSTS/MTG showed an enhanced response when auditory and visual object features were presented together, relative to presentation in a single modality. Finally, pSTS/MTG responded more to object identification than to other components of the behavioral task. We suggest that pSTS/MTG is specialized for integrating different types of information both within modalities (e.g., visual form, visual motion) and across modalities (auditory and visual).

MeSH terms

  • Acoustic Stimulation / methods*
  • Analysis of Variance
  • Auditory Perception / physiology
  • Cerebral Cortex / physiology
  • Humans
  • Magnetic Resonance Imaging / methods
  • Photic Stimulation / methods*
  • Temporal Lobe / physiology*
  • Visual Perception / physiology