Risk-dependent reward value signal in human prefrontal cortex

Abstract

When making choices under uncertainty, people usually consider both the expected value and risk of each option, and choose the one with the higher utility. Expected value increases the expected utility of an option for all individuals. Risk increases the utility of an option for risk-seeking individuals, but decreases it for risk averse individuals. In 2 separate experiments, one involving imperative (no-choice), the other choice situations, we investigated how predicted risk and expected value aggregate into a common reward signal in the human brain. Blood oxygen level dependent responses in lateral regions of the prefrontal cortex increased monotonically with increasing reward value in the absence of risk in both experiments. Risk enhanced these responses in risk-seeking participants, but reduced them in risk-averse participants. The aggregate value and risk responses in lateral prefrontal cortex contrasted with pure value signals independent of risk in the striatum. These results demonstrate an aggregate risk and value signal in the prefrontal cortex that would be compatible with basic assumptions underlying the mean-variance approach to utility.

Fig. 1.

Experimental design and behavioral measures of expected value and risk attitude. (A) Imperative task. Single stimuli were presented randomly in one of the 4 quadrants of a monitor for 1.5 s. Participants responded by indicating the quadrant in which stimuli appeared with a button press. Stimuli were associated with different combinations of reward magnitude and probability. (B) Choice task. A safe and a risky monetary choice option were presented randomly on the right and left side of a monitor for 5.5 s. Gambles with 2 levels of expected value and risk were used. The safe option on the left consists of a 100% gain of 45 £; the risky option on the right consists of a 50% gain of either 30 or 90 £(expected value = 60 £; 2 numbers are used for the safe option to keep visual stimulation comparable with the risky option). Participants chose an option with a button press on presentation of a go-signal. (C) Average change in pleasantness rating resulting from the imperative procedure in all participants as a function of expected value (imperative task; 15 participants, error bars represent SEM). The scale ranged from −5 (very unpleasant) to +5 (very pleasant). (D) Risk attitudes of single participants in experiment 1. (E) Average certainty equivalents of low and high expected value options with same risk in experiment 2 (14 participants, error bars represent SEM). (F) Risk attitudes of single participants in experiment 2. Risk attitude was measured as difference between certainty equivalents of the low- and high-risk options.

Fig. 2.

Integrated value and risk coding in lateral prefrontal cortex. (A) Value coding with safe options in experiment 1. Activation in lateral prefrontal cortex increasing with safe reward (peaks at 42/30/10 and 34/44/10; P < 0.05, small volume correction with false discovery rate in frontal lobe). Activations were identified by a linear contrast of the 3 safe options: 0, 100, and 200 points. (B) Increase of BOLD response with expected value in safe options irrespective of risk attitude in experiment 1. Peak activations from cluster shown in A covarying with expected value for both risk-averse and risk-seeking individuals (P < 0.05, simple regressions). The difference between the 2 groups was not significant (P = 0.97, unpaired t test). (C) Risk attitude-dependent activation or suppression of responses to risky options in lateral prefrontal cortex in imperative experiment 1. Time courses of responses were extracted from circled cluster shown in A. Responses to stimuli associated with different levels of risk (average variance of low- and high-risk stimuli = 2.5-k and 20-k points²) were averaged separately for risk-averse and risk-seeking participants and across the 4 levels of expected value of interest (50, 100, 150, and 200 points; average, 125 points). (D) Peak activations from time course analysis shown in A to different pairs of stimuli with same expected value but different risk, averaged separately for risk-averse and risk-seeking participants. Activations were higher with higher risk in risk-seeking participants, and lower in risk-averse participants (risk-averse participants: t = −8.6, P < 0.01; risk-seeking participants; t = 3.1, P < 0.05). Activations increased with expected values (left to right; for all: R² > 0.64, P < 0.06). The groups of participants were split according to the sign of risk attitude.

Fig. 3.

Control analyses. (A) Value coding with safe options in experiment 2. Lateral prefrontal cortex region with stronger activation for higher expected value (peak at 30/52/10; P < 0.05, small volume correction in 10-mm sphere around circled peak voxel at 34/44/10 shown in Fig. 2A defined by experiment 1). (B) Overlap of activations from experiment 1 and 2. To show the full extent of the overlap, activations from A and E were plotted at P < 0.05. Activations from experiment 2 were significant (P < 0.05, small volume correction) within significant voxels of experiment 1. (C and D) Correlation of risk attitude with coding of risk, but not expected value in lateral prefrontal cortex. For this analysis, we varied risk and kept expected value constant or varied expected value and kept risk constant. In clusters shown in A and in Fig. 2A, the high-risk option elicited stronger activation than the low-risk option in risk-seeking individuals, but weaker activation in risk-averse individuals. These activation differences correlated with the degree of individual risk aversion (open squares), in both imperative (C; experiment 1) and choice situations (D; experiment 2) (for experiment 1, the variance of low- and high-risk options was 7.5-k and 22.5-k points², the expected value of both options was 150 points; R² = 0.49, P < 0.01; for experiment 2, the variance of low- and high-risk options was 400 and 900 £², the expected value of both options was 60 £; R² = 0.81, P < 0.0001). Conversely, higher expected value elicited an increase in lateral prefrontal activation that did not covary with risk attitude (filled diamonds; for experiment 1, the expected value of low- and high-value options was 50 and 150 points, the variance of both options was 7.5-k points²; R² = 0.01, P = 0.64; difference between correlations: P = 0.05, z test; for experiment 2, the expected value of low- and high-value options was 30 and 60 £, the variance of both options was 400 £²; R² = 0.06, P = 0.39; difference between correlations: P < 0.01, z test). (E) Comparison of different risk attitude-weighted risk terms in lateral prefrontal cortex in experiment 1. Paired t tests revealed a significantly better fit of brain activation with risk-attitude weighted variance than skewness (t = 3.34, P < 0.01). The comparisons of skewness with coefficient of variation and SD did not reach significance (t = 1.74, P = 0.11 and t = 2.05, P = 0.06 for coefficient of variation and SD, respectively). To compare different risk terms, we normalized each one of them before multiplying it with negative individual risk attitude in experiment 1 and searching for covariation with brain activation. The analysis was restricted to risk-averse and risk-seeking participants (n = 12), because the equivalent regressors in risk neutral participants consisted only of zeros and, thus, did not discriminate between different risk terms. Error bars correspond to SEM.

Fig. 4.

Value coding irrespective of risk in striatum. (A) Activation in posterior striatum covarying with increasing expected value (peak at 18/-4/2; P < 0.05, small volume correction with false discovery rate in striatum). (B) Time courses of risk-independent responses in cluster shown in A, averaged separately for risk-averse and risk-seeking participants. Activations were averaged across the 4 levels of expected value (50, 100, 150, and 200 points; average, 125 points). (C) Activation in cluster shown in A to different pairs of stimuli with same expected value but different risk, averaged separately for risk-averse and risk-seeking participants. Peak activations were similar for low and high risk, irrespective of risk attitude (P > 0.66) and increased with expected value (left to right; for all 4 groups: R² > 0.86, P < 0.05). RA, risk averse; RS, risk seeking; L, low; H, high.