Prefrontal and striatal activity related to values of objects and locations

Abstract

The value of an object acquired by a particular action often determines the motivation to produce that action. Previous studies found neural signals related to the values of different objects or goods in the orbitofrontal cortex, while the values of outcomes expected from different actions are broadly represented in multiple brain areas implicated in movement planning. However, how the brain combines the values associated with various objects and the information about their locations is not known. In this study, we tested whether the neurons in the dorsolateral prefrontal cortex (DLPFC) and striatum in rhesus monkeys might contribute to translating the value signals between multiple frames of reference. Monkeys were trained to perform an oculomotor intertemporal choice in which the color of a saccade target and the number of its surrounding dots signaled the magnitude of reward and its delay, respectively. In both DLPFC and striatum, temporally discounted values (DVs) associated with specific target colors and locations were encoded by partially overlapping populations of neurons. In the DLPFC, the information about reward delays and DVs of rewards available from specific target locations emerged earlier than the corresponding signals for target colors. Similar results were reproduced by a simple network model built to compute DVs of rewards in different locations. Therefore, DLPFC might play an important role in estimating the values of different actions by combining the previously learned values of objects and their present locations.

Keywords: intertemporal choice; prefrontal cortex; reward; temporal discounting; utility.

Figure 1

Behavioral task and recording locations. (A) Temporal sequence of intertemporal choice task. (B) Recording locations for the neurons in the DLPFC that modulated their activity according to the difference in the temporally discounted values related to the targets in different positions (ΔDV_LR) or in different colors (ΔDV_RG). Horizontal (anterior-posterior) distance is measured relative to the arcuate sulcus, whereas the vertical (dorso-ventral) distance is relative to the principal sulcus. (C) Recording locations for the neurons recorded in the CD and VS. Oblique dotted lines indicate approximately the borders between the CD, VS, and putamen.

Figure 2

Three example DLPFC neurons with activity related to temporally discounted values (DVs). (A–C) A DLPFC neuron that modulated its activity according to the difference in the DVs for red and green targets. (A) Raster plots sorted according to the difference in the DVs between red (large reward) target and green (small reward) target (DV_Red − DV_Green). A pair of number to the left indicate reward delays for the red and green targets. Blue and black rasters show the action potentials during the trials in which the animal chose the small and large reward targets, respectively. Colored rectangles and vertical line segments indicate the trials grouped for the results shown in (B). (B) Spike density functions (SDF; top) and average firing rates during cue period (bottom) sorted by the difference in the DVs for red (large reward) and green (small reward) targets. Dotted (solid) lines and empty (filled) symbols represent SDF and average firing rates during the trials in which the animal chose the small (large) reward. (C) SDF and average firing rates of the same neuron shown in (A), estimated according to the difference in the DVs for left and right targets (represented in the grayscale). Dotted (solid) lines and empty (filled) symbols represent SDF and average firing rates during the trials in which the animal chose left (right) targets. (D–G) Average firing rates of two other DLPFC neurons. (D,E) Another neuron showing significant modulation in their activity related to the difference in the DVs for the large and small reward targets (D) and for the left and right targets (E). (F,G) A third neuron showing significant activity only for the difference in DVs of left and right targets. (D,E–G) are in the same format as in the bottom panel of (B,C). Error bars, SEM. *p < 0.01; **p < 0.001 (t-test for the corresponding regression coefficient).

Figure 3

Example neuron in the ventral striatum encoding the difference in the temporally discounted values for the red and green target. (A) Raster plots sorted according to the difference in the DVs between red and green targets. (B,C) Spike density functions and average firing rates during the cue period sorted by the difference in the DVs between red and green targets (B) and between left and right targets (C). Same format as in Figures 2A–C.

Figure 4

Neural activity related to temporally discounted values (DVs) in the DLPFC (A–C), CD (D–F), and VS (G–I). In (A,D,G), the standardized regression coefficients (SRC) associated with the difference in the DVs for left and right targets (ΔDV_LR = DV_Left − DV_Right) are plotted against the SRC associated with the difference in the DVs for red and green targets (ΔDV_RG = DV_Red − DV_Green). Empty symbols, neurons showing significant effects of ΔDV_RG. Black symbols, neurons showing significant effects of both variables. In (B,E,H), SRC for the DVs for red and green targets are plotted against each other. Black and empty symbols, neurons showing significant effect of DVs for both red and green targets and for only one of the two targets, respectively. In (C,F,I) SRC for the difference in the DVs for red and green targets during choice trials (ordinate) are plotted against the SRC for the difference in the fictitious DVs for red and green targets during control trials (FDV_Red − FDV_Green). Empty circles, neurons showing significant effect of ΔDV_RG and significant interaction between ΔDV_RG and the task (model 4); black filled symbols, neurons showing significant effect of ΔDV_RG without significant interaction with task.

Figure 5

Time course of DLPFC activity related to temporally discounted values (DVs) and reward delays in the DLPFC (A–D), CD (E–H), and VS (I–L). In (A,E,I), average coefficient of partial determination (CPD) calculated using a 200-ms sliding window is plotted separately for the difference in the DVs for the left and right targets (blue) and the difference for the red and green targets (purple). Frequency histograms in (B,F,G) show the latency for the neural activity related to the difference in the DVs. In (C,G,K), average CPD for the difference in the reward delays for the left and right targets (blue), the difference in the reward delays for the red and green targets (purple), and the position of the red (large reward) target (RM, red) are shown. Frequency histograms in (D,H,L) show the latency for the neural activity related to the difference in reward delays and the activity related to reward magnitude. Shaded areas, SEM.

Figure 6

Time course of neural activity related to different types of temporally discounted values (DVs) and the animal’s choice in DLPFC (A), CD (B), and VS (C). Coefficient of partial determination (CPD) was computed using a 200-ms sliding window separately for the activity related to the animal’s choice between the left and right target (C, green), its choice between red and green target (C*, gray), the difference in the DVs for the left and right targets (ΔDV_LR, blue), and the difference in the DVs for chosen and unchosen targets (ΔDV_CU, orange). Shaded areas, SEM.

Figure 7

A network model computing the temporally discounted values (DVs). (A) Pattern of connectivity within the model, with small circles and short line segments corresponding to excitatory and inhibitory connections, respectively. Units in the top layer receive the external inputs determined by the magnitude of reward expected for different target colors (red, large reward; green, small reward) and encode the DVs for red and green targets. Inputs to the middle layers (not shown) indicate the relative positions of red and green targets. For example, if the red target is presented to the right, then the inputs to units, Red-R and Green-L, are set high. Finally, the units in the bottom layer encode the DVs for left (blue) and right (gray) targets. (B) The activity of different units of the model during 4 sample trials in which the position of the large reward (red) target and its delay (0 or 8 s) were varied as indicated by the panels at the top (arrows indicating the target with the larger DV). Activity of different units is indicated by the same colors used in a. (C) Time course of activity change related to reward delay in the top (red unit) and bottom (blue/left unit) layers. This was given by the difference in the activity during the trials shown in the first and third column of (B).