Inverted activity patterns in ventromedial prefrontal cortex during value-guided decision-making in a less-is-more task

Abstract

Ventromedial prefrontal cortex has been linked to choice evaluation and decision-making in humans but understanding the role it plays is complicated by the fact that little is known about the corresponding area of the macaque brain. We recorded activity in macaques using functional magnetic resonance imaging during two very different value-guided decision-making tasks. In both cases ventromedial prefrontal cortex activity reflected subjective choice values during decision-making just as in humans but the relationship between the blood oxygen level-dependent signal and both decision-making and choice value was inverted and opposite to the relationship seen in humans. In order to test whether the ventromedial prefrontal cortex activity related to choice values is important for decision-making we conducted an additional lesion experiment; lesions that included the same ventromedial prefrontal cortex region disrupted normal subjective evaluation of choices during decision-making.

Fig. 1

Behavioral data—experiment 1. a Example stimulus-reward associations for HV, LV, and CV options. b Red shading indicates area of vmPFC/OFC lesion present in one or both animals. c Frequency of choosing objectively better of the two options. Controls (green bars) as well as lesioned animals (purple bars) preferred HV- to LV- stimuli and CV- to LV-stimuli. For the HV vs. CV decision, strikingly, controls preferred HV-stimuli, receiving only a subset of the rewards that CV-stimuli would have offered. However, macaques with vmPFC/OFC lesions did not prefer CV- to HV-stimuli as much as controls (right). d The pattern of results was replicated, including group difference in HV vs. CV decisions, even after HV vs. CV decisions were made easier by satiating animals with LV rewards prior to testing

Fig. 2

Behavioral data—experiment 2. a Example trials from fMRI experiment. After an inter-trial interval (ITI) visual stimuli, associated with different outcomes, are presented. Choices were followed, after a mean 4 s delay, with either the delivery of two juice drops (either LV or HV), four juice drops (CV comprising both LV and HV), or no reward. On each trial animals either chose between two stimuli (two-option trials) each associated with reward, in the example illustrated one stimulus is associated with a single reward (left-side stimulus) and the other with a compound reward (right-side stimulus) or, on some trials (one-option trials), a single stimulus was presented on one side of the screen and an animal could either choose it by touching the button placed in front of it or reject it by touching the button in front of the blank side of the screen. b Frequency of choosing the objectively better of the two options. Animals preferred HV- to LV- stimuli and CV- to LV- stimuli but they did not prefer CV- to HV-stimuli. c Reaction times (RTs) of choices between a CS+ (i.e., HV, CV, LV) and an unrewarded stimulus (the blank side of the screen or a CS−). RTs decreased with expected value of the reward

Fig. 3

VmPFC activity—experiment 2. a VmPFC activity increased at time of decision (top; cluster-corrected z > 2.3; p < 0.05; Supplementary Table 3: GLM-2, contrast 1). Activity increments were prominent in the area removed by the lesion in experiment 1 (bottom). b ROI time course illustrating the effect of decision on BOLD activity. Abscissa indicates time from decision onset and ordinate indicates beta regression coefficients relating the decision event to the BOLD signal. Coordinates for ROI correspond to green circle in a, top (3, 21, 3). (c) Left panel: estimates of each option’s value in each individual animal (M1–M4) and c Right panel: normalized choice values used in two GLM analyses of experiment 2. However, from left panel c it is clear that, prior to normalization, chosen values are usually higher than unchosen values. d Activity in vmPFC (1, 17, −2) covaried with the decision variable guiding choices—difference in subjective value (rather than objective reward amount) between choice taken and choice rejected (chosen value-unchosen value; cluster-corrected z > 2.3; p < 0.05 within 25 mm sphere centered on decision effect in a). The regression coefficients relating the BOLD signal to the difference between chosen and unchosen options at the time of choice is plotted in e and regression coefficients relating the BOLD signal to the value of the chosen option and the unchosen option are plotted separately in f. Difficulty increased, and vmPFC activity increased, when the chosen value was lower or the unchosen value was higher as shown in e and f. g Full summary of lesion location (first row) and of all activity in the frontal lobe positively and negatively related to taking a decision (second row: non-cluster-corrected results; third row: cluster-corrected results) and the decision variable used to guide the decision (chosen–unchosen option value difference; fourth row: non-cluster-corrected results; fifth row: cluster-corrected results). In summary, the results shown in a and d are representative of the pattern of activity found in adjacent regions and no negative decision-related activity and no positive value-related activity was observed within the lesioned areas

Fig. 4

Choice-based analysis of vmPFC activity—experiment 2. Time courses of vmPFC activity when HV, CV, LV, and unrewarded options are present (ROI from Fig. 3d). These are the four options whose values are illustrated in Fig. 3c (left panel). Because the impact that the option has on vmPFC activity changes depending on what other option is presented on any given trial (and therefore which option is likely to be taken and which is likely to be rejected) the time courses have been sorted by the value of the chosen option: HV a, CV b, or LV c. The effect of the value of the unchosen option can, for example, be seen in a: activity associated with choosing HV is greater when decisions are difficult and choices are made between it and CV (blue) as opposed to LV (green) or Unrewarded (red). The effect of the chosen value can be seen by comparing either the red lines or the green lines in a and b: activity associated with choosing an option increases when it is harder to make the choice because its value is lower. Although a–c show the time courses of the effects of the various options on vmPFC activity, d illustrates the same information but using the peaks of the time courses

Fig. 5

VmPFC activity—experiment 3. In experiment 3, each of the three options was associated with a drifting probability of reward. a Left panel: sigmoid functions illustrating the proportion of trials on which a stimulus (stimulus A) was chosen as a function of the difference between the values estimated for that stimulus and the alternative option (stimulus B). The value estimates were derived from a standard reinforcement learning algorithm (METHODS: Reinforcement learning—experiment 3). a Right panel: normalized choice values used in the GLM analysis. As in Fig. 3c (right panel) normalization was carried out separately on the chosen option value estimates, unchosen option value estimates, and the chosen–-unchosen value difference estimates. The effect shown in b is unpacked in c, demonstrating that activity in macaque vmPFC decreases as the value of the chosen option increases and increases as the value of the unchosen option increases. Note that in experiment 3 trials were performed quickly so that activity in the first seven seconds, approximately, reflects the current trial (trial n). Later activity reflects decisions on subsequent trials (trial n + 1). The gray vertical bar indicates the approximate boundary between trial n and n + 1. On 66% of occasions the option chosen on trial n would be offered again on trial n + 1 (and it was often chosen again) and on 66% of occasions the unchosen option on trial n would be offered again (in which case it was frequently unchosen again) and so the contrasts for trial n capture activity also on trial n+1