Predeliberation activity in prefrontal cortex and striatum and the prediction of subsequent value judgment

Abstract

Rational, value-based decision-making mandates selecting the option with highest subjective expected value after appropriate deliberation. We examined activity in the dorsolateral prefrontal cortex (DLPFC) and striatum of monkeys deciding between smaller, immediate rewards and larger, delayed ones. We previously found neurons that modulated their activity in this task according to the animal's choice, while it deliberated (choice neurons). Here we found neurons whose spiking activities were predictive of the spatial location of the selected target (spatial-bias neurons) or the size of the chosen reward (reward-bias neurons) before the onset of the cue presenting the decision-alternatives, and thus before rational deliberation could begin. Their predictive power increased as the values the animals associated with the two decision alternatives became more similar. The ventral striatum (VS) preferentially contained spatial-bias neurons; the caudate nucleus (CD) preferentially contained choice neurons. In contrast, the DLPFC contained significant numbers of all three neuron types, but choice neurons were not preferentially also bias neurons of either kind there, nor were spatial-bias neurons preferentially also choice neurons, and vice versa. We suggest a simple winner-take-all (WTA) circuit model to account for the dissociation of choice and bias neurons. The model reproduced our results and made additional predictions that were borne out empirically. Our data are compatible with the hypothesis that the DLPFC and striatum harbor dissociated neural populations that represent choices and predeliberation biases that are combined after cue onset; the bias neurons have a weaker effect on the ultimate decision than the choice neurons, so their influence is progressively apparent for trials where the values associated with the decision alternatives are increasingly similar.

Keywords: caudate nucleus; decision circuit-modeling; dorsolateral prefrontal cortex; free-choice decision making; monkey single-neuron recording; pre-deliberation decision bias; value-based decision-making; ventral striatum.

Figure 1

Task and behavior. (A) Three animals (two in the DLPFC experiment and one of the two and another in the basal-ganglia experiment) indicated their preference between a smaller and more immediate juice reward (designated by the green target) and a larger, but generally delayed one (red target). The number of yellow dots surrounding the targets—the clock—indicated the delay in seconds associated with each target. Portrayed are the full timelines and screenshots of two trials and the beginning of a third trial. Each trial is made up of a 1 s fore period, followed by a 1 s cue period, and afterwards a go signal leading to target fixation. Then, after the delay associated with the chosen target is counted down, the reward is delivered. And, after an inter-trial interval of 2 s, the next trial starts. If the animal chooses the target with the shorter delay, the difference between the longer and shorter delay is added to the inter-trial interval, so as not to motivate the animal to select the target with the shorter delay to bring about the next trial more quickly. The first trial depicts a choice between a 2 s-delay small reward and 8 s-delay large one, where the animal selects the large reward. In the second trial, the animal opts for an immediate small reward over a 2 s-delayed large reward. The interval between two ticks on the time scale is 1 s. The 1.5 s pre-cue period is denoted in light blue. (B) The mean probability (±SEM) of choosing the small reward vs. the 9 possible combinations of delays (N = 62 sessions) from both animals in the DLPFC, including the UNDET, WDET, and SDET probability ranges.

Figure 2

Three examples of DLPFC spatial-bias neurons and one non-bias neuron. Raster plots (top; ranked by increasing trial number for trials with subsequent leftward and rightward eye movements separately), spike-density function plots (mean ± SEM; middle), and logistic-regression best-fits (bottom) to the data of 4 DLPFC neurons (Panels A and D from Animal D, B and C from J). Cue onset is at 0 s, so depicted is spiking activity from the last half-second of the inter-trial interval as well as from the 1 s fore period. The data is taken from all the undetermined (UNDET) trials of each neuron, because bias neurons were selected using only UNDET trials. Panels (A–C) depict spatial bias neurons, while panel (D) is a non-bias neuron. The number of trials associated with each data point in the logistic regression is designated by the diameter of the symbol. (See Figure S2A for ROC and choice-probability analyses of these neurons.).

Figure 3

Two examples of DLPFC reward-bias neurons. Raster plots, spike-density function plots (mean ± SEM), and logistic-regression best-fits to the data of 2 DLPFC reward-bias neurons from all their UNDET trials (Panels A and B from Animal D and J, respectively). Details are given in Figure 2. (See Figure S2B for ROC and choice-probability analyses of these neurons.).

Figure 4

Two examples of VS spatial-bias neurons. Raster plots, spike-density function plots (mean ± SEM), and logistic-regression best-fits to the data of 2 VS reward-bias neurons from all their UNDET trials (Panels A and B from Animal H and J, respectively). Details are given in Figure 2. (See Figure S2C for ROC and choice-probability analyses of these neurons.).

Figure 5

Accuracy vs. chance level and proportion explained above chance level for DLPFC spatial- and reward-bias neurons and VS spatial-bias neurons. (A–C) Scatter plots of accuracy vs. chance level are depicted for UNDET trials. Bias neurons and anti-bias neurons (see Figure 6) are designated in red and cyan squares, respectively. (D) Average (±SEM) proportion explained above chance level over DLPFC spatial-bias neurons during UNDET, weakly determined (WDET), strongly determined (SDET), and no-choice control (NC) trials. The proportions explained for UNDET (0.16 ± 0.03; ±SEM) and WDET (0.03 ± 0.01) are significantly greater than expected by chance, while those at SDET (0.01 ± 0.01) and NC (−0.005 ± 0.004) are at chance (paired one-tailed t-test, p = 2 · 10⁻⁵, p = 0.01, p = 0.09, and p = 0.9, respectively). In addition, the slope of the regression line through the four trial types was significantly positive (p < 2 · 10⁻⁸). (E) Average (±SEM) proportion explained above chance level over DLPFC reward-bias neurons during UNDET, SDET NC trials. The proportion explained for UNDET (0.37 ± 0.04) was significantly greater than expected by chance while the SDET (0.03 ± 0.03) an NC (0.02 ± 0.01) ones were not (paired one-tailed t-test, p = 2 · 10⁻⁶, p = 0.17, and p = 0.06, respectively). The slope of the regression line through the data was also significantly positive (p < 4 · 10⁻⁷). WDET trials were not included because there were too few trials—only 5 of 12—for meaningful statistical analysis. (F) Average (±SEM) proportion explained above chance level over VS spatial-bias neurons during UNDET, SDET, and NC trials. The proportion explained for UNDET (0.17 ± 0.03) was significantly greater than expected by chance, while the SDET (−0.004 ± 0.01) and NC (−0.02 ± 0.01) values were not (paired one-tailed t-test, p = 8 · 10⁻⁵, p = 0.65, and p = 0.9, respectively). The slope of the regression line through the data was also significantly positive (p < 1 · 10⁻⁶). WDET trials were not included because there were too few trials—only 9 of 14—for meaningful statistical analysis. P-values in the figure are for one-tailed t-tests. (See Figure 6 for histograms of the percentiles of the prediction accuracies in the chance-level distributions.).

Figure 6

Histograms of percentiles of prediction accuracies in the distributions of chance level over the entire neuron populations. For each neuron in the population we calculated its prediction accuracy, and then by repeatedly and randomly shuffling its left/right or large-/small-reward labels also its chance-level distribution. We then computed in which percentile of its chance-level distribution the neuron's prediction accuracy lay. A histogram of these percentiles over all neurons is plotted here. Neurons at the top/bottom 5-percentile are designated bias/anti-bias neurons, respectively. (A) The mean percentile of all neurons involved in DLPFC spatial-bias analysis (n = 105) is 66% (green dotted line), and is significantly greater than chance-distribution mean (at 50%, black dotted line; Wilcoxon test p = 2 · 10⁻⁷). Also, 69 of the 105 DLPFC neurons are in the top 50-percentile of the chance-distribution mean, which is significantly more than expected by chance (binomial test p = 8 · 10⁻⁴). (B) The mean percentile of all neurons involved in DLPFC reward-bias analysis (n = 118) is 57% and significantly greater than the chance-distribution mean (Wilcoxon test p = 0.01); 70 of 118 neurons are in the top 50-percentile, significantly more than expected by chance (binomial test p = 0.026). (C) The mean percentile of all neurons involved in VS spatial-bias analysis (n = 52) is 63% and significantly greater than chance-distribution mean (Wilcoxon test p = 0.004); 33 of 52 neurons are in the top 50-percentile, significantly more than would be expected by chance (binomial test p = 0.035).

Figure 7

Pairwise intersections of spatial-bias neurons, reward-bias neurons, and choice neurons in the DLPFC. (A) The proportion of reward-bias neurons among spatial bias neurons is not significantly different than the proportion of reward-bias neurons among the entire neuronal population (p = 0.98, χ²-test). (B) The proportion of choice neurons among spatial-bias neurons is not significantly different than their proportion among the entire neuronal population (p = 0.73, χ²-test). (C) The proportion of choice neurons among reward-bias neurons is not significantly greater than their proportion among the entire neuronal population (p = 0.87, χ²-test).

Figure 8

Locations of neurons recorded in DLPFC and striatum. (A) Anatomical locations of the DLPC neurons. Distance from the cortical surface is not shown in this planar plot. Hence neurons that differ only in their depth appear on top of each other, as do neurons that belong to more than one group. No evidence was found for anatomical clustering of the various neural groups (1-way MANOVA p > 0.15 for all groups). (B) A coronal view of neuron locations in ventral striatum (VS; black marker edges) and caudate nucleus (CD; no marker edges). Dotted lines designate the border between CD, putamen, and VS.

Figure 9

A simple biased WTA circuit model accounts for the spatial-bias data and makes additional predictions. (A) A circuit for spatial-bias activity. Units x_L and x_R designate populations of left and right choice neurons, which are self-excitatory and mutually inhibitory (through inhibitory neuron h). Each choice unit is bi-directionally connected to a bias unit (p_L and p_R) Excitatory (inhibitory) connections are designated by arrows (circles). (B) Without bias input (b_L = b_R = 0), this is a simple WTA network, so for inputs I_R > I_L (I_R = 2 and I_L = 1.8, in this case) x_L's activity diminishes to zero and x_R's increases and then stabilizes. (C) When the left bias is turned on (b_L = 5, b_R = 0), x_L now wins for the inputs in B. Note that the bias activity was turned off (at t = 250 ms) shortly after the onset of the competition (at t = 200 ms). (D) A phase-space diagram simulating various values for b_L and I_L − I_R when b_R = 0 and I_R = 2. The separatrix between the basin of attraction of x_L winning and x_R winning is constant until approximately 1, when b_L passes its activation threshold, and then linear. (See Figure S3 for the full model, including both spatial- and reward-bias activity.).

Figure 10

The predictive power of spatial and reward DLPFC bias neurons and VS spatial-bias neurons during the pre-cue, cue and go periods. The average (±SEM) proportion explained above chance level and the average (±SEM) activity levels are depicted for the pre-cue, cue and go period for all 29 DLPFC spatial-bias neurons (in the left and right panels of A, respectively), for all 12 DLPFC reward-bias neurons (in the left and right panels of B, respectively), and for all 14 VS spatial-bias neurons (in the left and right panels of C, respectively). P-values in the figure are for one-tailed t-tests. (A) The proportion of accuracy explained in the pre-cue and go periods are not significantly different (one-tailed t-test p = 0.2), but are significantly larger than in the cue period, where the proportion drops to chance-level (one-tailed paired t-test p = 0.25). In contrast, the activity of the neurons in the go period is significantly weaker than in the pre-cue period. (B) The proportion explained for the pre-cue and cue periods are not significantly different (one-tailed t-test p = 0.5), but a significant decrease occurs for the go period, which is at chance level (one-tailed t-test p = 0.2). There is no significant trend in the activity levels among the pre-cue, cue, and go periods (one-way ANOVA p = 0.64). Note that the proportion explained for the DLPFC reward-bias pre-cue period is also significantly higher than the pre-cue proportion explained in DLPFC spatial bias (in A, one-tailed t-test p = 6 · 10⁻⁴). (C) The proportion explained drops significantly after the pre-cue period to chance level (one-tailed paired t-test p = 0.8 and p = 0.6 for the cue and go periods, respectively). Again there is no significant trend in the activity levels across the pre-cue, cue, and go periods (one-way ANOVA p = 0.3).