Rapid adaptation to auditory-visual spatial disparity

Abstract

The so-called ventriloquism aftereffect is a remarkable example of rapid adaptative changes in spatial localization caused by visual stimuli. After exposure to a consistent spatial disparity of auditory and visual stimuli, localization of sound sources is systematically shifted to correct for the deviation of the sound from visual positions during the previous adaptation period. In the present study, this aftereffect was induced by presenting, within 17 min, 1800 repetitive noise or pure-tone bursts in combination with synchronized, and 20 degrees disparate flashing light spots, in total darkness. Post-adaptive sound localization, measured by a method of manual pointing, was significantly shifted 2.4 degrees (noise), 3.1 degrees (1 kHz tones), or 5.8 degrees (4 kHz tones) compared with the pre-adaptation condition. There was no transfer across frequencies; that is, shifts in localization were insignificant when the frequencies used for adaptation and the post-adaptation localization test were different. It is hypothesized that these aftereffects may rely on shifts in neural representations of auditory space with respect to those of visual space, induced by intersensory spatial disparity, and may thus reflect a phenomenon of neural short-term plasticity.

Figure 1

Stimulus positions in Experiment 1. In the adaptation condition (A), sound stimuli were always 20° from visual stimuli. Positions of active loudspeakers and light-emitting diods (LEDs) were varied between trials. Immediately after completion of the adaptation condition, auditory (B) and visual localization (C) were tested by use of a pointing method. Trials with auditory and visual stimuli were presented in an alternating sequence.

Figure 2

Temporal sequence of stimuli and responses in adaptation and localization conditions. (A) In adaptation trials, the spatially disparate sound and light pulses were synchronized. Participants pressed a key as soon as the brightness of the visual stimulus was reduced. (B) In localization trials, participants pointed with a hand pointer toward the azimuthal position of the sound or light source. Participants pressed a key to indicate the final adjustment of the pointer.

Figure 3

Shifts in auditory and visual localization in two participants (A,B) after adaptation to 20° auditory-visual disparity (Experiment 1). Final pointer positions obtained after adaptation with the auditory stimuli presented to the right of the visual stimuli (+20° disparity) are plotted against the respective positions measured after adaptation with the auditory stimuli presented to the left of the visual stimuli (−20° disparity). Data for auditory (●) and visual localization (○) were fit by regression lines. The shift of the regression lines for auditory localization in both participants to the right of the diagonal (broken line) indicates that pointing responses to acoustic targets were more to the left after adaptation to +20° disparity than those obtained after adaptation to −20° disparity. A slight opposite trend was found for visual localization. Negative azimuths indicate stimuli to the left, and positive azimuths stimuli to the right.

Figure 4

Mean azimuthal angles (± SE) of the pointing responses for all participants, plotted as a function of stimulus azimuth (Experiment 1). Localization of auditory (A–C) and visual targets (D–F) is shown prior to adaptation (A,E), after adaptation with auditory stimuli to the left of the visual stimuli (−20°; B,E), and after adaptation with auditory stimuli to the right of the visual stimuli (+20°; C,F). Data were fit by regression lines. Parameters of the resulting function y = ax + b and coefficients of determination for each fit are as given in the panels. Negative angles are to the left, positive to the right.

Figure 5

Mean normalized shifts (± SE) in auditory (hatched bars) and visual localization (open bars), averaged over all stimulus positions, after adaptation to auditory-visual spatial disparity (Experiment 1; same data as in Fig. 4). Negative shifts are to the left, positive to the right. Asterisks next to bars indicate significant differences of pre- and post-adaptive localization; and asterisks next to brackets indicate significant differences of the respective auditory or visual shifts measured after adaptation to −20° and +20° disparity.

Figure 6

Mean normalized shifts (± SE) in auditory and visual localization after presentation of auditory and visual stimuli that were spatially aligned (Experiment 2). (A) Shifts measured after stimulus presentation to the right (0° R) and to the left (0° L) of the participant's median plane. (B) Shifts in the same participants as shown in A, obtained after adaptation to auditory-visual spatial disparity in Experiment 1. (C) Differences between normalized shifts measured in Experiments 1 (20° disparity) and 2 (0° disparity) with identical positions of visual stimuli. Conventions are as in Fig. 5. (Hatched bars) Auditory; (open bars) visual.

Figure 7

Mean normalized shift (± SE) in auditory (●) and visual localization (○) for one participant, plotted as a function of auditory-visual spatial disparity during adaptation. Results obtained after adaptation with auditory stimuli to the left and right of the visual stimuli are combined. Positive shifts indicate that localization is shifted toward the direction in which the auditory stimuli were presented during adaptation, and negative shifts indicate a shift to that side in which the visual stimuli have been presented during adaptation.

Figure 8

Mean normalized shifts (± SE) in localization of 1-kHz-tone and light pulses after adaptation to auditory-visual spatial disparity with either 1-kHz (A) or 4-kHz tone pulses (B). Conventions are as in Fig. 5.

Figure 9

Mean normalized shifts (± SE) in localization of 4-kHz-tone and light pulses after adaptation to auditory-visual spatial disparity with either 4-kHz (A) or 1-kHz tone pulses (B). Conventions are as in Fig. 5.