Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
, 4, 9

The Behavioral Relevance of Multisensory Neural Response Interactions


The Behavioral Relevance of Multisensory Neural Response Interactions

Holger F Sperdin et al. Front Neurosci.


Sensory information can interact to impact perception and behavior. Foods are appreciated according to their appearance, smell, taste and texture. Athletes and dancers combine visual, auditory, and somatosensory information to coordinate their movements. Under laboratory settings, detection and discrimination are likewise facilitated by multisensory signals. Research over the past several decades has shown that the requisite anatomy exists to support interactions between sensory systems in regions canonically designated as exclusively unisensory in their function and, more recently, that neural response interactions occur within these same regions, including even primary cortices and thalamic nuclei, at early post-stimulus latencies. Here, we review evidence concerning direct links between early, low-level neural response interactions and behavioral measures of multisensory integration.

Keywords: behavior; brain imaging; crossmodal; multisensory; neurophysiology.


Figure 1
Figure 1
Two potential varieties of multisensory interactions assessable by applying a linear model to the analysis of event-related potentials. The linear model involves comparing the summed responses from unisensory conditions with the response to the multisensory stimulus. In this figure, the level of activity (arbitrary units) within fictive brain regions is illustrated within the blue discs. In panel (A) modulations in response strength are illustrated, wherein the same set of brain regions is observed in response to the summed unisensory and multisensory conditions, albeit with greater magnitude in the latter case. This is illustrative of a supra-additive gain modulation. The colored topographic maps illustrate what one might observe in the ERP data. In panel (B) modulations in the configuration of brain regions active under multisensory stimulus conditions are illustrated, such that brain regions otherwise inactive under unisensory conditions are observed. In terms of event-related potential analyses, this latter mechanism would manifest as a modulation in the topography of the electric field at the scalp, which is illustrated in the voltage maps below. It should be noted that these two mechanisms, i.e., gain and generator modulations, can co-occur.
Figure 2
Figure 2
A schematic of the median split analysis approach applied in Sperdin et al. (2009). For each subject and stimulus condition, trials were sorted according to RT speed. Those with RTs faster than the median were considered “fast” and those slower than the median were considered “slow”. Event-related potentials were likewise separately averaged according to RT speed, and compared using the linear model schematized in Figure 1. Data were analyzed in using a multi-factorial within subjects design.
Figure 3
Figure 3
Evidence for the impact of early non-linear and supra-additive neural response interactions on RT speed. The top panels illustrate global field power waveforms in response to multisensory stimulus pairs, and the summed unisensory responses for trials producing fast and slow RTs (left and right panels, respectively). While non-linear neural response interactions began at 40 ms post-stimulus for trials producing fast RTs, such was only the case from 86 ms onwards for trials producing slow RTs. The middle portion illustrates the difference in source estimations over the 40–84 ms post-stimulus period between responses to multisensory stimulus pairs and summed unisensory responses for trials producing fast and slow RTs (red and green framed images, respectively). The sagittal slice is shown at x = −53 mm using the Talairach and Tournoux (1988) coordinate system. The bottom panel illustrates the mean scalar value of differential activity within a cluster of 25 solution points within the left superior temporal cortex. There were significantly greater non-linear multisensory neural response interactions in the case of trials producing fast RTs.

Similar articles

See all similar articles

Cited by 13 PubMed Central articles

See all "Cited by" articles


    1. Blau V., van Atteveldt N., Ekkebus M., Goebel R., Blomert L. (2009). Reduced neural integration of letters and speech sounds links phonological and reading deficits in adult dyslexia. Curr. Biol. 19, 503–50810.1016/j.cub.2009.01.065 - DOI - PubMed
    1. Britz J., Michel C. M. (2010). Errors can be related to pre-stimulus differences in ERP topography and their concomitant sources. Neuroimage 49, 2774–278210.1016/j.neuroimage.2009.10.033 - DOI - PubMed
    1. Busch N. A., Dubois J., VanRullen R. (2009). The phase of ongoing EEG oscillations predicts visual perception. J. Neurosci. 29, 7869–787610.1523/JNEUROSCI.0113-09.2009 - DOI - PMC - PubMed
    1. Cappe C., Barone P. (2005). Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey. Eur. J. Neurosci. 22, 2886–290210.1111/j.1460-9568.2005.04462.x - DOI - PubMed
    1. Cappe C., Morel A., Barone P., Rouiller E. M. (2009). The thalamocortical projection systems in primate: an anatomical support for multisensory and sensorimotor interplay. Cereb. Cortex 19, 2025–203710.1093/cercor/bhn228 - DOI - PMC - PubMed