Neural correlates of mentalizing-related computations during strategic interactions in humans

Abstract

Competing successfully against an intelligent adversary requires the ability to mentalize an opponent's state of mind to anticipate his/her future behavior. Although much is known about what brain regions are activated during mentalizing, the question of how this function is implemented has received little attention to date. Here we formulated a computational model describing the capacity to mentalize in games. We scanned human subjects with functional MRI while they participated in a simple two-player strategy game and correlated our model against the functional MRI data. Different model components captured activity in distinct parts of the mentalizing network. While medial prefrontal cortex tracked an individual's expectations given the degree of model-predicted influence, posterior superior temporal sulcus was found to correspond to an influence update signal, capturing the difference between expected and actual influence exerted. These results suggest dissociable contributions of different parts of the mentalizing network to the computations underlying higher-order strategizing in humans.

Fig. 1.

Inspection game and behavioral results. (A) Two interacting players are each individually given two action choices at the beginning of each trial. Players are given 1 second to respond, and their choices are highlighted with a red frame for another second before being covered with a blank screen. Five seconds after the start of the trial, the actions of both players are shown to each player for 1.5 s, with the payoff each one individually receives shown at the top. (B) Payoff matrix for the inspection game used in this paper. (C) Log likelihood errors for each computational model tested shows that the influence model, which incorporates the effects of players' actions influencing their opponents, has a better fit to subjects' behavior than either the RL or fictitious play models or these two models combined. To account for overfitting and the effects of differences in free parameters between models we used an out-of-sample prediction validation technique, as shown in Fig. S1. Error bars show the SEM of individual log likelihoods. (D) Furthermore, the actual probability of a player taking a specific behavioral action is linear with respect to the probability of choosing that action as computed by the influence model. Here, behavior and predictions are shown separately for the employer and employee. Error bars are SEM over subjects.

Fig. 2.

Expected reward signals. (A) At the time of choice, the expected reward of the action selected by a player is shown across the brain as calculated by different computational models. The expected reward signal from the influence model is correlated significantly with BOLD responses in mOFC (0, 36, −21 mm, z = 3.56), mPFC (−3, 63, 15 mm, z = 3.29), and in the right temporal pole (42, 15, −39, z = 3.98), the latter two areas surviving at P < 0.05 correction for small volume (SVC) within an 8-mm sphere centered on coordinates from areas implicated in mentalizing (3), whereas only the fictitious play model has significant activity in mOFC (at P < 0.001). The RL model had no significant activity correlating with expected reward anywhere in the brain. (B) An analysis to test for areas showing neural activity related to expected reward, which is explained significantly better by the influence model than by the RL model, revealed statistically significant effects in mPFC (−3, 57, 12 mm, z = 3.11; P < 0.05 SVC). (C) The average correlation coefficients for each model from the area reported in B (extracted from all voxels showing effects at P < 0.005 in mPFC). All images shown depict whole-brain voxel-wise comparisons; small volumes are defined only for the purposes of correction for multiple comparisons. (D) fMRI activity in mPFC shows a linear relation with binned expected reward probabilities as computed by the influence model (fMRI activity extracted from individual peaks in a 10-mm search radius centered on peak from B). (E) The computational models tested in this article make distinctly different predictions about the expected reward signals after switching actions (switch) or sticking to the same action (nonswitch) as a consequence of influencing the opponent. Intuitively, the underlying reason is that both RL and fictitious play will most likely “stay” after a reward and “switch” after a nonreward. However, the influence model has a higher incentive to switch even after receiving a reward. That is, expected reward signals associated with a specific action do not necessarily increase after the receipt of a reward when taking into consideration the influence that specific action exerts on the opponent's strategy. (F) fMRI responses in mPFC at the time of choice on switch compared with nonswitch trials show a response profile consistent with the influence model and not the fictitious play models or RL models (the data are extracted from a 10-mm sphere centered on peak from B). The difference between the employee and employer was significant at P = 0.02.

Fig. 3.

Influence signals in the brain. (A) At the time of outcome, the influence update of the inferred opponent's strategy shows significant correlations with activity in STS bilaterally (−57, −54, 0 mm, z = 3.32 and 60, −54, 9 mm, z = 3.35; P < 0.05 SVC). (B) The degree to which a subject thinks he/she is influencing his/her opponent can be measured by taking the difference in log-likelihood fits between the influence and fictitious models on each player's behavior. Likewise, brain regions invoked in computing the influence on the opponent will correlate more strongly with the influence model for subjects invoking this approach when compared with subjects that do not. Influence signals were found to significantly covary with the model likelihood difference (influence − fictitious) across subjects in mPFC (−3, 51, 24 mm, z = 4.09; P < 0.05 SVC). (Right) The relationship between influence regression coefficients and model likelihood differences in mPFC. All images shown depict whole-brain voxel-wise comparisons; small volumes are defined only for the purposes of correction for multiple comparisons.

Fig. 4.

Interregion correlation analysis. (A and B) Correlations among mPFC, STS, and ventral striatum were computed for each time point within a trial to determine whether there were significant changes in the correlations between these brain regions at the point the outcome was received (when prediction errors were generated) compared with other time points in the trial. Heat plots of region correlations through time are shown separately for each subject, with correlations between mPFC and STS shown in A and correlations between mPFC and ventral striatum (vStriatum) in B. Red indicates a high correlation between both regions, and blue indicates a low correlation. Shaded areas indicate the time subjects are given their choices (0 s, time of choice) and the time of outcome (5 s into trial). (C) The mean correlation between regions is shown averaged across all subjects for each time point in the trial. A significant increase in correlation after the outcome of a trial was revealed was found at 6.5 s into the trial (significant at P < 0.01 compared with the correlation at 4 s; paired t test). This supports the idea that information processed at the time of outcome in STS and ventral striatum is being shared with mPFC so as to facilitate updating in the expected reward for a given action. (D) Scatter plots of BOLD activity from a typical subject showing the correlation between regions after the trial outcome. Red lines indicates the linear regression fit of STS activity against mPFC activity (Left) and of ventral striatum activity against mPFC activity (Right).