Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain

Abstract

Midbrain dopaminergic neurons projecting to the ventral striatum code for reward magnitude and probability during reward anticipation and then indicate the difference between actual and predicted outcome. It has been questioned whether such a common system for the prediction and evaluation of reward exists in humans. Using functional magnetic resonance imaging and a guessing task in two large cohorts, we are able to confirm ventral striatal responses coding both reward probability and magnitude during anticipation, permitting the local computation of expected value (EV). However, the ventral striatum only represented the gain-related part of EV (EV+). At reward delivery, the same area shows a reward probability and magnitude-dependent prediction error signal, best modeled as the difference between actual outcome and EV+. In contrast, loss-related expected value (EV-) and the associated prediction error was represented in the amygdala. Thus, the ventral striatum and the amygdala distinctively process the value of a prediction and subsequently compute a prediction error for gains and losses, respectively. Therefore, a homeostatic balance of both systems might be important for generating adequate expectations under uncertainty. Prevalence of either part might render expectations more positive or negative, which could contribute to the pathophysiology of mood disorders like major depression.

Figure 1.

Stimulus layout for the guessing task, expected values, and associated prediction errors. a, b, Each guessing trial began with the presentation of the backside of eight playing cards (top). Initially, volunteers had to place money on individual playing cards. In some trials, money could be placed on the corners of four adjacent cards (a) and in others on a single card only (b). This manipulation allowed us to control reward probability. The cards were flipped 4.2 s after placing the bet (a, b, bottom). Seven of eight cards were black, and the remaining one was a red ace (a, b, bottom). If the red ace was selected (a), the volunteer gained the amount of money, and otherwise lost the money (b). c–g, Expected values and associated prediction errors in this paradigm. The actual values taken from Table 1 are convolved with a Gaussian for a better visual comparison with BOLD responses. The total expected value is shown in c. The gain-related expected value is depicted in d with the associated prediction error (e). The loss-related expected value is depicted in f with the associated prediction error (g). €, Euro.

Figure 2.

Activation during the anticipation of monetary rewards overlaid on a template T₁-weighted MR image at p < 0.001 (uncorrected). Parameter estimates from the FIR model (i.e., averaged activation time courses) for each peak voxel are plotted in bins of 2 s for all 42 volunteers. The figure background shows a grayscale-coded representation of the envelope of both expected hemodynamic responses (i.e., anticipation and outcome; high values, white; low values, dark). The bright (i.e., white) area ∼12 s after stimulus onset represents the predicted peak of the response evoked by the outcome related response; the gray plateau from ∼6–10 s is related to the BOLD response elicited by reward anticipation. a, Main effect of magnitude showing stronger BOLD signal changes for trials with five Euro as opposed to one Euro. Parameter estimates from the FIR model (i.e., time courses) are averaged across outcomes and reward probability, showing a clear main effect of reward magnitude (left ventral striatum x, y, z, −12, 3, 0 mm). b, Main effect of probability showing stronger BOLD signal for trials with high reward probability. Parameter estimates from the FIR model (i.e., time courses) are averaged across reward magnitude and outcomes (left ventral striatum x, y, z, −12, 15, −3 mm). 5 €, Five Euro. R, Right side of the brain.

Figure 3.

Activations expressing gain-related expected value overlaid on a template T₁-weighted MR image at p < 0.001 (uncorrected). a, Bilateral ventral striatum shows BOLD signal changes covarying with a linear model of EV⁺. Parameter estimates from the FIR model (i.e., averaged activation time courses) for the right-sided peak (x, y, z, 12, 9, −3 mm) in 42 volunteers. b, In addition, we replicated this analysis in a second cohort of volunteers (n = 24). As in the first cohort, bilateral ventral striatum shows BOLD signal changes covarying with a linear model of EV⁺. The parameter estimates of the FIR model (i.e., activation time courses) are for the right-sided peak (x, y, z, 12, 9, −3 mm) for the second cohort (n = 24). R, Right side of the brain.

Figure 4.

Ventral striatal activation showing EV⁺-related activation during anticipation and an EV⁺-related prediction error during outcome (conjunction analysis) overlaid on a coronal T₁-weighted MR image at p < 0.001 (uncorrected). a, Bilateral ventral striatum coexpressed EV⁺-related predictions and the ensuing prediction error. b, This was replicated in a second cohort (n = 24). c, Parameter estimates from the FIR model (i.e., averaged activation time courses) for five-Euro trials are plotted in bins of 2 s from the peak voxel (x, y, z, −12, 6, −3 mm) for the first cohort (n = 42) (a). The gray shades in the background are pictorial representations of the expected hemodynamic responses for anticipation and outcome (compare Fig. 2). 5 €, Five Euro. R, Right side of the brain.

Figure 5.

Responses in the amygdala in relationship to loss-related expected value and the associated prediction error (conjunction analysis) at p < 0.001. a, Bilateral amygdala showed BOLD signal changes that resembled EV⁻ during reward anticipation and an EV⁻-based prediction error signal during the outcome phase. b, This was replicated in a second cohort (n = 24). c, Parameter estimates from the FIR model (i.e., averaged activation time courses) for five Euro trials are shown for the right amygdala (x, y, z, 27, 0, −15) in bins of 2 s for 42 volunteers. The gray shades in the background are pictorial representations of the expected hemodynamic responses for anticipation and outcome (compare Fig. 2). 5 €, Five Euro. R, Right side of the brain.

Figure 6.

a, Dynamic probabilities during the experiment. Each point on the dashed line denotes the gain probability up to this trial for the low (red; one card selected) and high probability (green; 4 cards selected) trials. The solid line depicts the overall probability (i.e., as calculated over all trials). b, Ventral striatal activation showing EV⁺-related activation during anticipation and EV⁺-related prediction error during outcome overlaid on a coronal T₁-weighted MR image at p < 0.001 (uncorrected). The left part shows the activation pattern from Figure 5a, using the actual gain probabilities (26 and 66%) over the course of the experiment. The right panel shows the result as obtained when using the time varying probabilities from the model depicted in a. c, The same analysis for EV⁻-related activation patterns (compare Fig. 5). 5 €, Five Euro. R, Right side of the brain.