How We Transmit Memories to Other Brains: Constructing Shared Neural Representations Via Communication

Abstract

Humans are able to mentally construct an episode when listening to another person's recollection, even though they themselves did not experience the events. However, it is unknown how strongly the neural patterns elicited by mental construction resemble those found in the brain of the individual who experienced the original events. Using fMRI and a verbal communication task, we traced how neural patterns associated with viewing specific scenes in a movie are encoded, recalled, and then transferred to a group of naïve listeners. By comparing neural patterns across the 3 conditions, we report, for the first time, that event-specific neural patterns observed in the default mode network are shared across the encoding, recall, and construction of the same real-life episode. This study uncovers the intimate correspondences between memory encoding and event construction, and highlights the essential role our common language plays in the process of transmitting one's memories to other brains.

Keywords: communication; memory; mental construction; naturalistic; pattern similarity.

Figure 1.

Circle of communication. Depiction of the entire procedure during sharing of an experience. Participants encode the movie and then reinstate it during recall. By listening to the audio recall, listeners construct the movie events in their mind. Mental representations related to the movie are shared throughout this cycle and transmitted across the brains via communication.

Figure 2.

Experiment design and analysis. (A) 18 participants watched a 25-min audio–visual movie (Merlin) while undergoing fMRI scanning (movie-viewing). One participant separately watched the movie and then recalled it inside the fMRI scanner and her spoken description of the movie was recorded (spoken-recall). Then a group of 18 participants who were naïve to the content of the movie listened to the recorded narrative. The entire procedure was repeated with a second movie (Sherlock) by recruiting new groups of participants. (B) Depiction of the length of each event in the movie (y-axis) relative to its corresponding event (if remembered) in spoken-recall (x-axis) for each movie. Each box denotes a different event. Boxes that are out of the continuous diagonal string of events depict the events that were recalled in an order different from their original place in the movie. (C) Schematic for the main analysis. Brain data were averaged within each scene in the dataset of each condition (e.g., condition x = movie-viewing and condition y = spoken-recall). Averaging resulted in a single pattern of brain response across the brain for each scene for each condition. Then these 2 patterns were compared and correlated using a searchlight method. Significant values were computed by shuffling the scene labels and comparing the nonmatching scenes. Similar analyses were performed for all other comparisons (spoken-recall to listening, listening to movie-viewing).

Figure 3.

Movie-viewing to spoken-recall pattern similarity analysis (A,B). Pattern similarity searchlight map, showing regions with significant between-participant, scene-specific correlations (P values) between spoken-recall and movie-viewing (searchlight was a 5 × 5 × 5 voxel cube). Panel A depicts data for the Merlin movie and panel B depicts data for the Sherlock movie. Dotted circle shows the approximate location of the PCC ROI that was used in the analysis in (C,D). Pattern similarity (r values) of each participant's encoding (movie-viewing) data to the brain response during spoken-recall (in the speaker) in posterior cingulate cortex. Red bar shows average correlation of matching scenes and blue bar depict average correlation of nonmatching scenes, averaged across subjects. Circles depict values for individual subjects. Panel C depicts data for the Merlin movie and panel D depicts data for the Sherlock movie.

Figure 4.

Spoken-recall to listening pattern similarity analysis (A,B). Pattern similarity searchlight map, showing regions with significant between-participant, scene-specific correlations (P values) between spoken-recall and listening (searchlight was a 5 × 5 × 5 voxel cube). Panel A depicts data for the Merlin movie and panel B depicts data for the Sherlock movie. Dotted circle shows the approximate location of the PCC ROI that was used in the analysis in (C,D). Pattern similarity (r values) of each participant's listening data to the brain response during spoken-recall (in the speaker) in posterior cingulate cortex. Red bar show average correlation of matching scenes and blue bar depict average correlation of nonmatching scenes, averaged across subjects. Circles depict values for individual subjects. Panel C depicts data for the Merlin movie and panel D depicts data for the Sherlock movie.

Figure 5.

Listening to movie-viewing pattern similarity analysis (A,B). Pattern similarity searchlight map, showing regions with significant between-participant, scene-specific correlations (P-values) between movie-viewing and listening (searchlight was a 5 × 5 × 5 voxel cube). Panel A depicts data for the Merlin movie and panel B depicts data for the Sherlock movie. Dotted circle shows the approximate location of the PCC ROI that was used in the analysis in panel (C,D). Pattern similarity (r values) of each participant's movie-viewing data to the average of all other listeners in posterior cingulate cortex. Red bar show average correlation of matching scenes and blue bar depict average correlation of nonmatching scenes, averaged across subjects. Circles depict values for individual subjects. Panel C depicts data for the Merlin movie and panel D depicts data for the Sherlock movie.

Figure 6.

Pattern similarity of movie viewers to listeners—relationship to behavior. (A,C) Correlation between the comprehension score of listeners and degree of similarity between the speaker neural responses during the encoding phase (i.e., while watching the movie) and all listeners in PCC, for each movie. (B,D) Rank order correlation values of the same analysis as in A,C for each of the viewers (including the speaker, red circle). Note that correlating the listeners’ brain responses with the actual speaker's brain responses during encoding phase better predicted comprehension levels than the correlation with other viewers.

Figure 7.

Shared neural patterns across all conditions. Regions showing scene-specific pattern correlations across movie-viewing, spoken-recall, and listening for (A) the Merlin movie and (B) the Sherlock movie.