Influence of talker discontinuity on cortical dynamics of auditory spatial attention

Abstract

In everyday acoustic scenes, listeners face the challenge of selectively attending to a sound source and maintaining attention on that source long enough to extract meaning. This task is made more daunting by frequent perceptual discontinuities in the acoustic scene: talkers move in space and conversations switch from one speaker to another in a background of many other sources. The inherent dynamics of such switches directly impact our ability to sustain attention. Here we asked how discontinuity in talker voice affects the ability to focus auditory attention to sounds from a particular location as well as neural correlates of underlying processes. During electroencephalography recordings, listeners attended to a stream of spoken syllables from one direction while ignoring distracting syllables from a different talker from the opposite hemifield. On some trials, the talker switched locations in the middle of the streams, creating a discontinuity. This switch disrupted attentional modulation of cortical responses; specifically, event-related potentials evoked by syllables in the to-be-attended direction were suppressed and power in alpha oscillations (8-12 Hz) were reduced following the discontinuity. Importantly, at an individual level, the ability to maintain attention to a target stream and report its content, despite the discontinuity, correlates with the magnitude of the disruption of these cortical responses. These results have implications for understanding cortical mechanisms supporting attention. The changes in the cortical responses may serve as a predictor of how well individuals can communicate in complex acoustic scenes and may help in the development of assistive devices and interventions to aid clinical populations.

Keywords: Alpha lateralization; Auditory attention; Event-related potentials; Neural oscillations.

Figure 1:

(A) Trial design. Each trial started with a visual cue to indicate the side to be attended. The cue was followed by a fixation dot at the center of the screen, then the stimulus presentation. Following the stimulus, the response screen was shown, prompting the listener for a response. Feedback was provided on each trial. (B) Two streams of CV were presented on each trial, one spoken by a male and the other by a female speaker. The streams were separated using interaural time differences corresponding to approximately ± 30°. In the continuous trials, the talker at each location remained the same. In contrast, in the switch trials, the two talkers swapped locations in the third CV presentation. (C) The stimulus timing was designed to allow isolation of the ERPs for each CV. The trial began with a noise-burst, indicated in black, followed by the start of the leading/target stream. The lagging/masker stream began 0.18 s after the leading stream, creating an asynchrony in the CV onsets. The colored envelope superimposed on the plot represents the talker at that location. (D) Scalp topography of the N1 response to the first target CV. White circles indicate the electrodes used for ERP analysis.

Figure 2:

(A) Behavioral performance for each condition. The black whisker plots show population results with horizontal lines indicating across-subject medians; error bars depict the maximum and minimum percent correct observed in each condition. Results for individual listeners are indicated by circles, with gray lines connecting results in the two conditions. ***P<0.001. (B) Error rates as a function of target CV position in trials with only a single target.

Figure 3:

(A) Grand average epoched EEG response for the active listening continuous (black) and switch (red) trials along with example topographies for each trial type. Vertical grey lines indicate the N1 of CVs in the leading/target stream, while the orange lines indicate the N1s of the CVs in the lagging/distractor stream. The yellow highlighted region indicates the time of the CVs following the switch in talkers, while the light blue highlighted region shows the time of the CVs after the switch. Topographies present the scalp distribution of N1 amplitude for the fourth CV in the leading stream in the to-be-attended continuous, and to-be-attended switch trials. (B) Grand average epoched EEG response for the passive continuous (dashed black) and switch (dashed red) trials. Topographies represent the scalp distribution of N1 amplitude for the third CV in the leading stream in the passive listening continuous and switch trials. (C) Average peak N1 amplitude across subjects for each CV in the target stream for the passive (open box) and active (filled box) conditions. A more negative value on the ordinate indicates a larger N1. Lines in each box plot indicate the median. Highlights correspond to the switch and post-switch CVs, as in panel A and B. *P<0.05, **P<0.01.

Figure 4:

Power in the alpha band, as a function of time, compared across conditions. The highlighted region in blue represents the time window in which the alpha power was significantly reduced in the switch trials relative to the continuous trials. *P<0.05 after adjustment for multiple comparisons. Dashed lines indicate the onset of CVs in the target stream. The scalp topography of the average difference in alpha power between switch and continuous trials is shown on the right over the blue-highlighted time window where the difference reached statistical significance.

Figure 5:

Topographic maps of the AMIα in two time periods (before and after a potential switch in talker) for continuous (A) and switch (B) trials. Bar graphs show mean across the posterior half of channels (excluding frontal channels) on the left hemisphere (LH) and right hemisphere (RH). Error bars indicate ±1 SEM. AMIα showed a significant hemispheric lateralization (LH>RH) in both conditions before a potential switch. This lateralization remained significant in the second time window in the continuous trials where the talker remained in the same location (A: right panel). In contrast, when the talker switched location in the switch trials, the lateralization pattern was disrupted and was no longer significant. *P<0.05; **P<0.01; n.s., not significant.

Figure 6:

Relationship between the behavioral cost of talker discontinuity, defined as (% correct in Continuous- % correct in Switch trials), and (A) the difference in the N1 in continuous vs. switch (larger negative values indicate larger suppression of the N1 in the switch trials, corresponding to greater neural disruption of attention) and (B) the decrease in power in the alpha band, both calculated in a time window immediately following the switch in talker. Dashed lines represent 90% confidence intervals. *P<0.05.