Intermodal auditory, visual, and tactile attention modulates early stages of neural processing

Abstract

We used event-related potentials (ERPs) and gamma band oscillatory responses (GBRs) to examine whether intermodal attention operates early in the auditory, visual, and tactile modalities. To control for the effects of spatial attention, we spatially coregistered all stimuli and varied the attended modality across counterbalanced blocks in an intermodal selection task. In each block, participants selectively responded to either auditory, visual, or vibrotactile stimuli from the stream of intermodal events. Auditory and visual ERPs were modulated at the latencies of early cortical processing, but attention manifested later for tactile ERPs. For ERPs, auditory processing was modulated at the latency of the Na (29 msec), which indexes early cortical or thalamocortical processing and the subsequent P1 (90 msec) ERP components. Visual processing was modulated at the latency of the early phase of the C1 (62-72 msec) thought to be generated in the primary visual cortex and the subsequent P1 and N1 (176 msec). Tactile processing was modulated at the latency of the N160 (165 msec) likely generated in the secondary association cortex. Intermodal attention enhanced early sensory GBRs for all three modalities: auditory (onset 57 msec), visual (onset 47 msec), and tactile (onset 27 msec). Together, these results suggest that intermodal attention enhances neural processing relatively early in the sensory stream independent from differential effects of spatial and intramodal selective attention.

Figure 1

Schematic Diagram of Experimental Apparatus. A) Auditory, vibrotactile, and visual stimuli were spatially coregistered to the left or right of fixation and participants responded to targets with the opposite hand. The stimulated hand was placed behind a sound permeable screen onto which visual stimuli were flashed via a DLP projector. B) The visual stimuli, speaker, and vibration device were aligned in azimuth and elevation. Stimulus waveforms for the auditory and tactile modalities are shown in the legend.

Figure 2

Electrode Groups. Electrodes groups were defined surrounding electrodes of interest for each component. Electrode groups on the midline are labeled according to the first letter of the center electrode (e.g. FZ → F). Electrode groups contralateral to the side of stimulation are indicated with a “c”.

Figure 3

Intermodal Attention Effect on Visual ERPs. A) ERPs in response to visual stimuli when the visual modality is attended (Red) or not attended (Blue) are shown in two midline electrode groups (C and P) and two posterior-lateral electrode groups (P-i, ipsilateral to stimulation and P10-c, contralateral). The difference wave between attended and unattended visual ERPs is overlaid (Black). The C1 and N1 were modulated with visual attention (p < 0.05) with a 100 ms baseline; the C1, P1 and N1 modulation were significant with a 50 ms baseline. B) A posterior view of the scalp topographies of the attention effects (Attend – Unattend) are shown at the latencies of the C1, P1, and N1. Smaller insets show the topography of each component of the average ERP across all visual stimuli (attended and unattended).

Figure 4

Intermodal Attention Effect on Auditory ERPs. A) ERPs in response to auditory stimuli when the auditory modality is attended (Red) or not attended (Blue) are shown in two midline electrode groups at frontal (F) and central (C) sites. The difference wave between attended and unattended auditory ERPs is overlaid (Black). The inset of group F emphasizes the modulation of the Na and P1 with attention (p < 0.05). B) Scalp topographies of the attention effect (Attend – Unattend) are shown at the latency of the Na and P1. The top row depicts the scalp contralateral to the side of stimulation, the bottom, ipsilateral. Smaller insets show the topography for the average ERP across all auditory stimuli (attended and unattended).

Figure 5

Intermodal Attention effect on Vibrotactile ERPs. A) ERPs in response to vibrotactile stimuli when the tactile modality is attended (Red) or not attended (Blue) are shown in two midline electrodes, frontocentral (FC) and posterior (P) and two frontal-lateral electrodes (FC-i, ipsilateral to stimulation and FC-c, contralateral). The difference wave between attended and unattented is overlaid (Black). The first significant effect with attention was at the latency of the N160 (p < 0.05) in electrode group FC-c. B) Scalp topographies of the attention effect (Attend – Unattend) are shown at the latency of the N160. The smaller insets show the topography of the N160 across all vibrotactile stimuli (attended and unattended).

Figure 6

Intermodal attention effects on gamma band oscillatory responses (GBRs). The left panel shows the ERSP images of the difference between attended and unattended ERSPs in the gamma band for visual (top panel), auditory (middle panel), and vibrotactile (bottom panel) stimuli averaged for channels indicated in white (at right). Outermost contours indicate a significant difference between attended and unattended (p < 0.05 one-tailed, inner contours are drawn at intervals decreasing by 0.01). The plot below each ERSP image shows the average power within the frequency range of the attention effect over time. Green lines indicate the on- and offset of significance. Attention effects in the gamma band onset at 47 ms for the visual modality, 57 ms for the auditory modality, and 27 ms for the vibrotactile modality. The right panel shows the scalp topographies of the significant gamma band attention effect.