Free choice shapes normalized value signals in medial orbitofrontal cortex

Abstract

Normalization is a common cortical computation widely observed in sensory perception, but its importance in perception of reward value and decision making remains largely unknown. We examined (1) whether normalized value signals occur in the orbitofrontal cortex (OFC) and (2) whether changes in behavioral task context influence the normalized representation of value. We record medial OFC (mOFC) single neuron activity in awake-behaving monkeys during a reward-guided lottery task. mOFC neurons signal the relative values of options via a divisive normalization function when animals freely choose between alternatives. The normalization model, however, performed poorly in a variant of the task where only one of the two possible choice options yields a reward and the other was certain not to yield a reward (so called: "forced choice"). The existence of such context-specific value normalization may suggest that the mOFC contributes valuation signals critical for economic decision making when meaningful alternative options are available.

Fig. 1

Lottery task and choice behavior. a A sequence of events in free choice trials. Pie charts indicated reward magnitudes from 60 to 600 μl in 60 μl increments. Gray color of the central fixation target indicated that the monkeys could choose either option freely. In the forced choice trials (red or yellow fixation color), monkeys were required to choose the color-matched target among the alternatives, unless otherwise the trials were aborted. Positions of the risky and safe options were fixed during a single payoff block. Gray bars (top) indicate the 1.0 s time periods used to analyze neuronal activity; cue, saccade and feedback periods. b Payoff matrix: in each payoff block 1 to 4, the monkeys chose between a 100% fixed amount of water reward and a lottery that would deliver a reward with 50% probability (5 different risky reward magnitudes per one block). For example, in payoff block 1 (PB1), the safe 60 μl reward was represented by a 1/10 filled pie chart and the risky option was represented by a pie chart ranging from empty to 4/10 full. c An example payoff block sequence (randomly selected without replacement until all four payoffs were presented). In a block the first 36 trials were forced choice trials. Then, 50 free choice trials (10 of each type) followed in random order. d Percentages (P) of risky choice plotted against magnitude of risky reward in each PB (indicated by color). Dashed colored lines indicate where risky and safe options have equal expected value

Fig. 2

Relative value signals in the activity of mOFC neurons. a Rasters and histograms of an example mOFC neuron modulated by the relative value of options. The activity aligned at cue onset during free choice trials was shown for 20 lottery pairs (four PBs times five LPs, 200 trials). Black dots in the histograms indicate raster of spikes. Gray bars indicate the cue period to estimate the neuronal firing rates shown in b. SAC indicate approximate time of saccade onset. b Activity plot of the mOFC neuron in a against the expected values of risky (EVr) and safe option (EVs). Error bars indicate s.e.m. The neuron showed positive and negative regression coefficients for EVr and EVs (EVr+EVs− type, EVr, 0.042, EVs, −0.048, AIC = 1283), respectively. c Activity histogram of 15 mOFC neurons modulated by relative values of risk and safe options during cue period (EVr+EVs– type). Activity in each of four payoff blocks (PB1–4) is shown for the five types of lottery pairs (LP1–5). d Percentage of mOFC neurons modulated by relative values during three task periods. Gray indicates activity showing the positive and negative regression coefficients for EVr and EVs, respectively (EVr+EVs− type). White indicates activity showing negative and positive regression coefficients for EVr and EVs, respectively (EVr−EVs+ type)

Fig. 3

Potential normalized value coding models. Schematic depiction of predicted neuronal responses in the four alternative normalized value coding models. In each panel, four colored lines indicated the model output (y-axis) in each of payoff block (PB1–4) plotted against the expected values of risky option (x-axis). Expected values of safe option were 60, 120, 180 and 240 μl in PB1 to 4, respectively. Model equations are shown in each plot. R_max, β, σ, b and G were free parameters. For this schematic drawing, the following values for free parameters were used; 1. Advanced fractional model, R_max, β and σ were 40 spk s⁻¹, 20 and 10 μl, respectively; 2. Simple fractional model, R_max and b were 40 and 10 spk s⁻¹, respectively; 3. Difference model, G and b were 0.4 (a.u.) and 10 spk s⁻¹, respectively; 4. Range normalization model, R_max and b were 40 and 10 spk s⁻¹, respectively. See Methods for more details

Fig. 4

The advanced fractional normalization model best explained mOFC relative value coding. a Four model outputs fit to the example neuronal activity encoding relative value (same neuron as shown in Fig. 2a). Average firing rates and s.e.m. in 20 lottery pairs are plotted in each panel. Colored lines indicate the best-fit lines segregated by payoff block. b Plots of the AIC differences between models estimated across the population. Mean and s.e.m. were estimated for 81 activities that showed relative value coding. AIC differences between model 1 and the other three relative expected value models are shown. c Same as b, but for AIC differences between model 1 and alternative models 5–10. See Methods for details of the models. Asterisk indicates statistical significance of the AIC differences from zero at P < 0.01 using one sample t-test

Fig. 5

Comparisons of the model performances for relative value coding. a Plots of the percent variance explained by the four normalization model for the mean response-based data estimated in 20 lottery pairs. b Same as a, but for the single trial-based data. c Plots of the percent variance explained by the models for the mean response-based data when cross-validation was performed. Percent variance explained for training data and test data are shown

Fig. 6

Comparison of the estimated normalization parameters and observed firing rates. a–c Box plots of the estimated parameters in the advanced fractional model. The R_max, β, and σ were plotted separately during three task periods. d Plots of the maximal firing rate observed in each mOFC neurons against the estimated R_max. e Plots of the baseline firing rate observed in each mOFC neurons against the model output with no value information (R_max β σ⁻¹). Dashed lines in d, e indicate regression slopes. Correlation coefficients and statistical significance are shown

Fig. 7

Attenuated value coding of mOFC neurons during forced choice trials. a Plots of the absolute value of regression coefficients for EVr (gray) and EVs (white) during free and forced choice trials. Mean ± s.e.m. during free and forced choice trials: EVr, 0.031 ± 0.002, free choice, 0.017 ± 0.002, forced choice; EVs, 0.042 ± 0.003, free choice, 0.027 ± 0.003, forced choice. b Average of the absolute value of regression coefficients for EVr and EVs across the trial block. Regression coefficients were estimated every 12 trials from the start of the payoff block. Error bars indicate s.e.m. c Activity plots of the same neuron in Fig. 4 during the forced choice trials. Color lines indicated the best-fit lines during the forced choice trials. Gray lines indicated the best-fit lines during the free choice trials as shown in Fig. 4a. d–f Probability density of the estimated parameters of the models during forced choice trials (brown), the 1st half of the free choice trials (green), and 2nd half of the free choice trials (blue). In d–f, triangles in the figures indicate the median. g Plots of the AIC differences between models estimated across the population. AIC differences between model 11 and models 12–13 are shown. Error bars indicate s.e.m. In a–g, the results during forced choice trials were shown when assuming expected values of risky and safe options were defined as indicated by pie chart stimuli (assumption 1, see Methods for details)