Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2014 Oct 28;111(43):E4687-96.
doi: 10.1073/pnas.1323812111. Epub 2014 Sep 29.

Coupled Neural Systems Underlie the Production and Comprehension of Naturalistic Narrative Speech

Affiliations
Free PMC article

Coupled Neural Systems Underlie the Production and Comprehension of Naturalistic Narrative Speech

Lauren J Silbert et al. Proc Natl Acad Sci U S A. .
Free PMC article

Abstract

Neuroimaging studies of language have typically focused on either production or comprehension of single speech utterances such as syllables, words, or sentences. In this study we used a new approach to functional MRI acquisition and analysis to characterize the neural responses during production and comprehension of complex real-life speech. First, using a time-warp based intrasubject correlation method, we identified all areas that are reliably activated in the brains of speakers telling a 15-min-long narrative. Next, we identified areas that are reliably activated in the brains of listeners as they comprehended that same narrative. This allowed us to identify networks of brain regions specific to production and comprehension, as well as those that are shared between the two processes. The results indicate that production of a real-life narrative is not localized to the left hemisphere but recruits an extensive bilateral network, which overlaps extensively with the comprehension system. Moreover, by directly comparing the neural activity time courses during production and comprehension of the same narrative we were able to identify not only the spatial overlap of activity but also areas in which the neural activity is coupled across the speaker's and listener's brains during production and comprehension of the same narrative. We demonstrate widespread bilateral coupling between production- and comprehension-related processing within both linguistic and nonlinguistic areas, exposing the surprising extent of shared processes across the two systems.

Keywords: brain-to-brain coupling; intersubject correlation; speech comprehension; speech production.

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Fig. 1.
Fig. 1.
Measuring natural speech production. (A) The experimental design involves a speaker first telling a spontaneous story inside an fMRI scanner (reference speaker, R) and then retelling the same story inside the scanner (S1–S9). For illustration we present a 2-min segment of the audio traces for the original production (R, reference) and two reproductions of the story (S1 and S2). Lines between the audio traces indicate differences in the timing of the same utterances across recordings. (B) The time-warp analysis stretches and compresses subsections of one audio recording to maximize its correlation to the first reference (R) recording, resulting in a time-warp vector of maximally correlated time points unique to each repetition of the story (see blue and red diagonal lines for S1 and S2, respectively). The zoom (Inset) of a segment of time-warp vector shows the deviation of the time-warp vector from the identity matrix. (C) The resultant vector is used to interpolate the audio recordings to equal lengths. (D) The time-warped audio envelopes show strong zero-lag cross-correlation (r = 0.68 ± 0.09 SE), whereas the linear-interpolated audio envelopes are more weakly correlated (r = 0.18 ± 0.10 SE). This demonstrates the efficacy of time warping for temporally aligning the audio signals across recordings.
Fig. 2.
Fig. 2.
Time warping the fMRI signal. (A) For illustration we present raw fMRI time courses from a given voxel in the motor cortex (MC) during speech production of the same story. (B) The time-warp vectors generated by time warping a given audio signal to the original (reference) recording (from Fig. 1B) are used to transform the fMRI signals measured during story retellings to a common time base. (C) Each fMRI response is interpolated individually according to the time-warping template of its corresponding audio envelope, thus improving the alignment between the brain responses and the envelope of the audio across repetitions of the story. (D) The time-warped fMRI signals show strong zero-lag cross-correlation (r = 0.20 ± 0.05 SE), whereas the linear-interpolated audio envelopes are more weakly correlated (r = 0.07 ± 0.05 SE). This demonstrates the efficacy of time warping for temporally aligning the fMRI time courses across recordings.
Fig. 3.
Fig. 3.
(A) Areas that exhibit reliable neural responses across runs (n = 9) during which the first primary speaker produced a 15-min real-life story. The results are presented on lateral and medial views of inflated brains and one sagittal slice at Talairach coordinate x = 13. (B) Areas that exhibit reliable responses during the production of the same 15-min story between the primary speaker and secondary speaker 1. (C) Areas that exhibit reliable responses during the production of the same 15-min story between the primary speaker and secondary speaker 2. Anatomical abbreviations: AG, angular gyrus; BG, basal ganglia; CS, central sulcus; IFG, inferior frontal gyrus; mPFC, medial prefrontal cortex; PCC, posterior cingulate cortex; Prec, precuneus; STG, superior temporal gyrus; TPJ, temporal–parietal junction. See Table S1 for a complete list of areas that respond reliably during speech production.
Fig. 4.
Fig. 4.
Areas in which the responses during speech production are coupled to the responses during speech comprehension. The comprehension–production coupling includes bilateral temporal cortices and linguistic and extralinguistic brain areas (see Table S1 for a complete list). Significance was assessed using a nonparametric permutation procedure and the map was corrected for multiple comparisons using an FDR procedure.
Fig. 5.
Fig. 5.
Schematic summary of the networks of brain areas active during real-life communication. Areas that exhibited reliable time courses only during the production of speech are marked in red and include the right and left motor cortex, right premotor cortex, left anterior section of the IFG, right anterior inferior temporal (IT), and the caudate nucleus of the striatum. Areas that exhibited reliable time courses only during the comprehension of speech are marked in yellow, and include the right and left IPS, the left and right posterior STG, and the right anterior IFG. Areas that exhibited reliable time courses during both the production and comprehension of speech (overlapping areas) but in which the response time courses during the production and comprehension of speech did not correlate are marked in orange. These areas include sections of the left and right MTG, sections of the left and right IPS, and the PCC. Areas in which the response time courses during the production and comprehension of speech are coupled are marked in blue. These areas include comprehension related areas along the left and right anterior and posterior STG, left anterior and posterior MTG, left and right TP, left and right AG, and bilateral TPJ; production-related areas in the dorsal posterior section of the left IFG, the left and right insula, and the left premotor cortex; and a collection of extra linguistic areas in the precuneus and medial prefrontal cortices.

Comment in

Similar articles

See all similar articles

Cited by 51 articles

See all "Cited by" articles

Publication types

LinkOut - more resources

Feedback