Decoding the neural substrates of reward-related decision making with functional MRI

Abstract

Although previous studies have implicated a diverse set of brain regions in reward-related decision making, it is not yet known which of these regions contain information that directly reflects a decision. Here, we measured brain activity using functional MRI in a group of subjects while they performed a simple reward-based decision-making task: probabilistic reversal-learning. We recorded brain activity from nine distinct regions of interest previously implicated in decision making and separated out local spatially distributed signals in each region from global differences in signal. Using a multivariate analysis approach, we determined the extent to which global and local signals could be used to decode subjects' subsequent behavioral choice, based on their brain activity on the preceding trial. We found that subjects' decisions could be decoded to a high level of accuracy on the basis of both local and global signals even before they were required to make a choice, and even before they knew which physical action would be required. Furthermore, the combined signals from three specific brain areas (anterior cingulate cortex, medial prefrontal cortex, and ventral striatum) were found to provide all of the information sufficient to decode subjects' decisions out of all of the regions we studied. These findings implicate a specific network of regions in encoding information relevant to subsequent behavioral choice.

Fig. 1.

Task outline and classifier construction. (A) Reversal task setup. Subjects chose one of two fractals, which on each trial were randomly placed to the left or right of the fixation cross. The chosen stimulus is illuminated until 2 s after the trial onset. After a further 1 s, a reward (winning 25 cents) or punishment (losing 25 cents) is delivered for 1 s, with the total money earned displayed at the top. The screen is then cleared, and a central fixation cross is presented for 8 s before the next trial begins. One stimulus is designated the correct stimulus, in that choosing that stimulus leads to a monetary reward on 70% of occasions and a monetary loss 30% of the time. The other stimulus is “incorrect,” in that choosing that stimulus leads to a reward 40% of the time and a punishment 60% of the time. After subjects choose the correct stimulus on four consecutive occasions, the contingencies reverse with a probability of 0.25 on each successive trial. Subjects have to infer that the reversal took place and switch their choice, at which point the process is repeated. The last three scans in a trial are used by our classifier to decode whether subjects will switch their choice or not in the next trial. A canonical BOLD response elicited at the time of reward receipt is shown (in green) to illustrate the time points in the trial at which the hemodynamic response is sampled for decoding purposes. A new trial was triggered every 12 s to ensure adequate separation of hemodynamic signals related to choices on consecutive trials. The average of three scans between the outcome of reward and the time of choice in the next trial was used for decoding subjects' behavioral choice in the next trial. These three time points will not only contain activity from the decision itself (activity taking place after the receipt of feedback, but before the next trial) but also activity from the reward/punishment received in the current trial and activity consequent to the choice made in the current trial. (B) The multivariate region classifier used in this study is divided in two parts. The first extracts a representative signal from each region of interest (Left) by averaging the brain voxels within a region weighted by the voxels' discriminability of the switch vs. stay conditions. To avoid overfitting the fMRI data, we did not take into consideration the correlations between voxels within a region of interest (Eq. 3). The second part of the classifier (Right) adds up the signal from each region, weighted by the region's importance in classifying the subject's decision (Eq. 2). Weights are calculated by using a multivariate classifier that uses each region's decoding strength, and correlations between regions, to maximize the accuracy of the classifier in decoding whether subjects are going to switch or stay (see Discriminative Analysis).

Fig. 2.

Global and local fMRI signals related to behavioral choice. (A) Here, we show fMRI signals related to behavioral choice, i.e., whether subjects will switch or maintain (stay) their choices on a subsequent trial. Voxel t-scores for the discriminability between switch and stay trials is shown for two individual subjects with data in its original form (Left) and then decomposed into a global spatial component (with spatial scale >8 mm; Center) and a local spatial component (spatial scale <8 mm; Right). The ACC region of interest is outlined in white for reference. Red and yellow indicate increased responses on switch compared with stay trials, whereas blue colors indicate stronger responses on stay compared with switch trials (also see SI Fig. 10). (B) Results from a group random effects analysis across subjects conducted separately for the original unsmoothed data, global data, and local data. Whereas global signals survive at the random effects level (consistent with classical fMRI analyses), local spatial signals do not survive at the group random effects level. Random effect t-scores are shown with a threshold set at P < 0.2 for visualization.

Fig. 3.

Illustration of the decoding accuracy for subjects' subsequent behavioral choices for each individual region and combination across regions. (A) Plot of average accuracies across subjects shown separately for local and global spatial scales. Both spatial scales contain information that can be used to decode subjects' subsequent behavioral choice, in all of our regions of interest. Notably, decoding accuracies are comparable at the local and global scales within each region. (B) Plot of average across accuracy across subjects for each region individually combining both local and global signals. (C) Results of the hierarchical multiregion classifier analysis, averaged across subjects. An ordering of regions was performed by starting with a classifier that only contains the individual region with best overall accuracy (ACC; leftmost column), and iteratively adding to this classifier the regions whose inclusion increases the accuracy of the classifier the most (or decreases the least). Thus, the second column shows the accuracy of a classifier containing ACC and ventral striatum, the third column the accuracy of a classifier containing ACC, ventral striatum, and mPFC, and so forth. The combination of the three regions that provide the best decoding accuracy are highlighted in gray. Addition of a fourth region (dorsal striatum) does not significantly increase decoding accuracy. All error bars indicate standard errors of the mean. (D) Decoding accuracy for the three region classifier shown separately for each individual subject (also see SI Table 1).