Nucleus accumbens neurons encode predicted and ongoing reward costs in rats

Abstract

Efficient decision-making requires that animals consider both the benefits and the costs of potential actions, such as the amount of effort or temporal delay involved in reward seeking. The nucleus accumbens (NAc) has been implicated in the ability to choose between options with different costs and overcome high costs when necessary, but it is not clear how NAc processing contributes to this role. Here, neuronal activity in the rat NAc was monitored using multi-neuron electrophysiology during two cost-based decision tasks in which either reward effort or reward delay was manipulated. In each task, distinct visual cues predicted high-value (low effort/immediate) and low-value (high effort/delayed) rewards. After training, animals exhibited a behavioral preference for high-value rewards, yet overcame high costs when necessary to obtain rewards. Electrophysiological analysis indicated that a subgroup of NAc neurons exhibited phasic increases in firing rate during cue presentations. In the effort-based decision task (but not the delay-based task), this population reflected the cost-discounted value of the future response. In contrast, other subgroups of cells were activated during response initiation or reward delivery, but activity did not differ on the basis of reward cost. Finally, another population of cells exhibited sustained changes in firing rate while animals completed high-effort requirements or waited for delayed rewards. These findings are consistent with previous reports that implicate NAc function in reward prediction and behavioral allocation during reward-seeking behavior, and suggest a mechanism by which NAc activity contributes to both cost-based decisions and actual cost expenditure.

Figure 1

Task design and behavioral results. (a,b) Schematic representing effort-based (a) or delay-based (b) decision tasks. On forced-choice low cost/immediate reward trials (left panels), a cue light was presented for 5s and was followed by extension of two response levers into the behavioral chamber. A single lever press (FR1) on the lever corresponding to the cue light led to immediate reward (45 mg sucrose) delivery in a centrally located food receptacle. Responding on the other lever did not produce reward delivery and terminated the trial. On forced choice high cost/delayed reward trials (middle panels), the other cue light was presented for 5s before lever extension. On these trials, a reward was delivered after either sixteen responses (FR16, effort based decision task) or a delay (FR1 + 4 or 8s delay, delay based decision task). Responses on the opposite lever terminated the trial and no reward was delivered. On free choice trials (left panels), both cues were presented simultaneously, and animals could select either response option. After training, NAc electrophysiological activity was monitored in vivo during a single 90-trial behavioral session. (c-e) Behavioral performance in the effort-based task. (c) Percentage of possible rewards obtained on forced-choice trials. Animals overcame high effort requirements to maximize rewards. (d) Percentage of errors on forced-choice trials were significantly below chance levels (50%; p < 0.0001 for both comparisons), demonstrating behavioral discrimination between cues. (e) Response allocation on free-choice trials, as percentage of choices. Dashed line indicates behavioral indifference point (i.e., the lack of a preference). Animals robustly preferred the low cost option (* p < 0.0001). (f-h) Behavioral performance in the delay-based task. (f) Percentage of possible rewards obtained on forced-choice trials. For both trial types, animals obtained nearly all available rewards. (g) Percentage of errors on forced-choice trials were significantly below chance levels (50%; p < 0.0001 for both comparisons), demonstrating behavioral discrimination between cues. (h) Response allocation on free-choice trials. Animals robustly preferred the immediate reward option (* p < 0.0001).

Figure 2

NAc neurons are activated during different components of the task. (a) Peri-event histogram (PEH) and raster plots of a representative cue-activated NAc neuron. Data shown are from high cost trials in the effort-based task, and are aligned to cue onset (left panel), the initial lever press response (middle panel), and reward delivery (right panel). Red circles in raster indicate timing of reward delivery. (b) Venn diagrams illustrating proportion of neurons that exhibited excitations following cue onset for the effort- and delay-based tasks. Insets show proportion of total cells that were activated by cues. Lower diagrams show number of cells that responded to the high value cue (gold), low value cue (blue), or both cues (overlap). (c) PEH and raster plots of a representative press-activated NAc neuron. Data shown are from delayed reward trials in the delay-based task. (d) Venn diagrams illustrating proportion of neurons that exhibited excitations preceding the lever press for the effort- and delay-based tasks. (e) PEH and raster plots of a representative reward-activated NAc neuron. Data shown are from high cost trials in the effort-based task. (f) Venn diagrams illustrating proportion of neurons that exhibited excitations following reward delivery for the effort- and delay-based tasks.

Figure 3

Cue-activated neurons exhibit selective responses that encode future reward cost but not reward delay. (a) PEHs of representative low cost selective (left panel), high cost selective (middle panel), and non-selective (right panel) cue-activated NAc neurons during the effort task. Data are aligned to cue onset (black bar). (b) Mean activity (±SEM) of cue-activated NAc neurons on low and high cost trials. Diagonal line represents purely non-selective activity. (c) Differential activity of all cue-activated NAc neurons in the effort task (low cost minus high cost). Each circle represents a single neuron, with low cost selective neurons shown in gold, non-selective neurons in black, and high cost selective neurons in blue. As a population, these neurons were significantly biased towards the low cost option (p = 0.04). (d) PEHs of representative immediate reward selective (left panel), delayed reward selective (middle panel), and non-selective (right panel) cue-activated NAc neurons during the delay task. (e) Mean activity (±SEM) of cue-activated NAc neurons on immediate and delayed reward trials. Diagonal line represents purely non-selective activity. (f) Differential activity of all cue-activated NAc neurons in the delay task (immediate minus delayed reward). Conventions follow from panel c. As a population, these neurons were not significantly selective for either option (p = 0.97).

Figure 4

Low-cost selective neurons in the effort task reflect action-specific reward value. (a) PEHs of a representative low cost selective neuron on forced choice (left panel) and free choice (right panel) trials. Conventions follow from Fig. 3a. Free choice trials involve the presentation of both cues, but were subdivided into trials in which the animal selected the low cost option or the high cost option. On forced choice trials, the cue-evoked increase in activity was smaller when the high cost option was subsequently chosen. (b) Mean (±SEM) activity traces of all low cost selective neurons (from Fig. 3c) on free and forced choice trials. (c) Comparison of activity following cue onset on forced choice and free choice trials. In both cases, activity was greater when the low cost choice was selected than when the high cost choice was selected (p < 0.01 for both comparisons).

Figure 5

Response- and reward-related activations do not encode effort or delay. (a) Differential activity plots for neurons activated prior to the response, after the initial response, prior to response completion/reward delivery, and following response completion/reward delivery in the effort-based task. Gray area in top panels illustrate the 2.5s epoch under consideration, relative to lever press responses on low cost and high cost trials (gold and blue tick marks, respectively). Lower panels represent differential activity for each neuron, as in Fig. 3c. Overall activity was not significantly cue-selective for any epoch (all p’s > 0.1). (b) Differential activity plots for neurons activated prior to the response, after the response, prior to reward delivery, and following reward delivery in the delay-based task. Conventions follow from panel a, except that tick marks in top panels represent lever press responses on immediate and delayed reward trials. Again, activity was not significantly cue-selective for any epoch (all p’s > 0.5).

Figure 6

A subset of NAc neurons is activated throughout reward seeking or delay period. (a-b) PEH and raster plots from a representative reward-seeking activated NAc neuron on low and high cost trials. For both, data are aligned to cue onset, and the black triangle denotes lever extension (at 5s). Trials in raster plots are sorted based on the latency between lever extension and reward delivery (red circles). (c) Mean (± SEM) Z-score of 22 neurons that were excited throughout execution of response requirements on high cost trials. Data are aligned to cue onset (left panel), the initial response (center panel), and reward delivery (right panel). On high cost trials, activity was maintained until reward delivery. (d-e) PEH and raster plots from a single representative delay-activated NAc neuron on immediate and delayed reward trials. Blue circles in raster indicate lever press responses. Other conventions follow from panel a. (f) Mean (± SEM) Z-score of 11 neurons that were activated during the delay period on delayed reward trials. Data are aligned to cue onset (left panel), the lever press response (center panel), and reward delivery (right panel). On delayed reward trials, activity was maintained until reward delivery.

Figure 7

Neurons inhibited prior to lever press responses exhibit prolonged responses on high cost or delayed reward trials. All conventions follow from Fig. 6. (a-b) PEH and raster plots from a representative response-inhibited NAc neuron on low and high cost trials. (c) Mean (± SEM) Z-score of 58 neurons that were inhibited before the initial response in the effort task. On high cost trials, activity was significantly lower than baseline both following the initial response and prior to reward delivery (*p < 0.01). (d-e) PEH and raster plots from a representative response-inhibited NAc neuron on immediate and delayed reward trials. (f) Mean (± SEM) Z-score of 35 neurons that were inhibited before the lever press response in the delay task. On delayed reward trials, activity was significantly lower than baseline both following the lever press response and prior to reward delivery (*p < 0.01).

Figure 8

Successive coronal diagrams illustrating anatomical distribution of electrode locations across core and shell of the NAc for the effort-based task (left) and delay-based task (right). Marked locations are limited to electrodes that contributed to data presented here. Filled circles indicate electrode location in the NAc core, open circles indicate electrode locations in the NAc shell. Numbers indicate anteroposterior coordinates rostral to bregma (in mm).