Evidence accumulation for value computation in the prefrontal cortex during decision making

Abstract

A key step of decision making is to determine the value associated with each option. The evaluation process often depends on the accumulation of evidence from multiple sources, which may arrive at different times. How evidence is accumulated for value computation in the brain during decision making has not been well studied. To address this problem, we trained rhesus monkeys to perform a decision-making task in which they had to make eye movement choices between two targets, whose reward probabilities had to be determined with the combined evidence from four sequentially presented visual stimuli. We studied the encoding of the reward probabilities associated with the stimuli and the eye movements in the orbitofrontal (OFC) and the dorsolateral prefrontal (DLPFC) cortices during the decision process. We found that the OFC neurons encoded the reward probability associated with individual pieces of evidence in the stimulus domain. Importantly, the representation of the reward probability in the OFC was transient, and the OFC did not encode the reward probability associated with the combined evidence from multiple stimuli. The computation of the combined reward probabilities was observed only in the DLPFC and only in the action domain. Furthermore, the reward probability encoding in the DLPFC exhibited an asymmetric pattern of mixed selectivity that supported the computation of the stimulus-to-action transition of reward information. Our results reveal that the OFC and the DLPFC play distinct roles in the value computation during evidence accumulation.

Keywords: decision making; prefrontal cortex; value.

Fig. 1.

(A) Task design. Four shapes were presented sequentially on the computer screen while the monkey fixated at the FP. After the FP was turned off, the monkey made a saccade to either the red or the green choice target. The shapes were selected randomly with replacement in each trial from a set of 10. Each of them was assigned a unique weight (Inset). The reward probability was calculated with the sum of the weights associated with the four shapes. Positive weights indicated that the red target had a higher reward probability than the green one. The red, yellow, green, and blue shadings on the time axis indicate the four stimulus representation periods. (B) Monkey performance. The percentage of red choices is plotted against the summed weight of the four shapes. Each dot indicates the percentage of red choice in trials with the same grouped summed weights. The curve is a logistic function fitted to all trials. (C) Subjective weights. We used logistic regression to assess the leverage of each shape on the monkeys’ choices and defined the coefficients as the subjective weights. Positive subjective weights indicate a tendency to choose the red target. (D) Two competing decision-making hypotheses. In stimulus-based decision making, the evidence regarding color (Wc) is first accumulated into the ΣWc, which is then used to generate the action-independent color choice and finally forms the action. In contrast, the action-based decision-making hypothesis assumes that the Wc is first transformed into the action domain (Wa), which is then accumulated into the ΣWa and used to form the action. In addition, the stimulus-based decision making may be complemented by a motor preparation process, in which the ΣWa is calculated from the ΣWc during decision making.

Fig. 2.

Example neurons. (A) The activity of an OFC neuron encoded the Wc. Response averages are aligned to the shape onsets and extended 100 ms into the next shape period. The trials are divided into quartiles by the Wc (indicated by the color), and the neuron’s response averages are computed. The dashed vertical lines indicate the shape offset at 300 ms. (B) The activity of a DLPFC neuron encoded the ΣWa. The trials are divided into quartiles by the ΣWa (indicated by the redness), and the neuron’s response averages are computed. Other conventions are the same as in A. See also SI Appendix, Fig. S6 for the neurons’ selectivities to the other variables.

Fig. 3.

Choice representations in (A) the OFC and (B) the DLPFC. The average firing rate difference of the neurons between trials with different spatial and color choices are shown. Green, spatial choice; yellow, color choice; black, shuffled data. The green and yellow shaded areas indicate the SEM, and the gray shaded area indicates the SD of the 1,000 shuffles. The green and yellow horizontal lines mark the periods in which the spatial or color choice curve is significantly different from the shuffled data (P < 0.01, one-way ANOVA). The four color-shaded boxes indicate the shape presentation period in the four epochs, with the dashed lines indicating the shape onset. The rightmost dashed line marks the average saccade time. The gray bar on the horizontal axis represents the period from which the mean firing rates were calculated to define the neurons’ preferred choices (Methods).

Fig. 4.

Representations of the single weights and the summed weights in the OFC with LASSO. The effect sizes of (A) Wc, (B) Wa, (C) ΣWc, (D) ΣWa, and (E) choice were quantified using normalized |SRβ|. (A–D) The red, yellow, green, and blue traces indicate the weights associated with the first, second, third, and fourth epochs, respectively. (E) Green and yellow traces indicate spatial choice and color choice, respectively. The solid sections of each curve indicate significance (P < 0.01 for at least a consecutive 150 ms), and the dashed sections are not significant. Shaded areas along the curves stand for the SEM. The traces of the ΣWc and ΣWa of the first epoch were omitted in C and D (Methods).

Fig. 5.

Representations of the single weights and the summed weights in the DLPFC with LASSO. The effect sizes of (A) Wc, (B) Wa, (C) ΣWc, (D) ΣWa, and (E) choice were quantified using normalized |SRβ|. (A–D) The red, yellow, green, and blue traces indicate the weights associated with the first, second, third, and fourth epochs, respectively. (E) Green and yellow traces indicate spatial choice and color choice, respectively. The solid sections of each curve indicate significance (required P < 0.01 for at least a consecutive 150 ms), and the dashed sections are not significant. Shaded areas along the curves stand for the SEM. The traces of the ΣWc and ΣWa of the first epoch were omitted in C and D (Methods).

Fig. 6.

Mixed selectivity. (A) An example DLPFC neuron showing nonlinear mixed selectivity to the spatial configuration and the color weights. A multiple-regression analysis indicated that the effect on spike counts induced by the interaction term of configuration and weight was significant (P = 0.002, t test). (B) The distribution of the VDI of the DLPFC (Left, blue) and the OFC (Right, red) neurons and their corresponding shuffled distributions (gray). Solid lines indicate fitted probability distributions based on KDE (Methods). Colored triangles on the x axes indicate the medians of the corresponding distributions. The VDIs of DLPFC neurons are significantly larger than the shuffled data (P = 0.002, Kolmogorov–Smirnov test), while the VDIs of OFC neurons are not (P = 0.567, Kolmogorov–Smirnov test).