Shaping of motor responses by incentive values through the basal ganglia

Abstract

The striatum is a key neural interface for cognitive and motor information processing in which associations between reward value and visual stimulus can be used to modify motor commands. It can guide action-selection processes that occur farther downstream in the basal ganglia (BG) circuit, by encoding the reward value of an action. Here, we report on the study of simultaneously recorded neurons in the dorsal striatum (input stage of the BG) and the internal pallidum (output stage of the BG) in two monkeys performing a center-out motor task in which the visual targets were associated with different reward probabilities. We show that the tuning curves of motor-related neurons in both structures are modulated by the value of the action before movement initiation and during its execution. The representations of values associated with different actions change dynamically during the task in the internal globus pallidus, with a significant increase in the number of encoding neurons for the chosen target at the onset of movement. This report sheds additional light on the functional differences between the input and output structures of the BG and supports the assertion that the dorsal basal ganglia are involved in movement-related decision-making processes based on incentive values.

Figure 1.

Behavioral paradigm. A, The reward probability-based free-choice task. Two targets (P1 and P2) are displayed simultaneously during each trial, in four possible positions in random order (6 target combinations) and in random locations (4 × 3 possibilities). B, Combinations of displayed targets during the task. C, Example of movement trajectories executed during 50 successful trials by monkey T.

Figure 2.

Behavioral results. A, Probability of choosing target P1, according to its relative reward value. P1 and P2 were target electronic labels with no visible differentiation between the two as perceived by the monkeys. The monkeys optimized their strategy by choosing cues associated with the highest reward probabilities. B, Mean and SD of the RT and MD of successful trials. The gray and white columns represent monkeys T and D, respectively. The ratio PC/(PC+PNC) represents the degree of choice complexity between the possibly chosen and the nonchosen cue probabilities.

Figure 3.

Population perievent histograms. n refers to the number of neurons for each population selected according to the corresponding regression. A, Neural discrimination of movement direction. The encoding of movement direction was modeled by a tuning curve (nonlinear regression analysis, R² > 0.8; p < 0.05). The superimposed responses were related to PD, +90, +180, and + 270°. GPi neurons fired according to target location during excitatory (GPi+) or inhibitory (GPi−) response peaks to task epochs. Below each plate, the distribution (mean ± SD) for the gain (g) of the individual tuning curves is provided. B, Representative reward–value encoding neurons. Probability encoding is achieved by means of a monotonic variation in firing rate, with increasing reward probability (linear regression analysis, R² > 0.8; p < 0.05). This encoding can be characterized by an increasing (positive slope) or decreasing (negative slope) firing rate. Below each plate, the distribution (mean ± SD) for the slope (a) of the individual linear regressions is provided.

Figure 4.

Theoretical reconstruction of recording sites. The dots show the estimated locations of the electrode tips during recording sessions (scale in millimeters). These are identified according to the encoded parameter (movement direction, reward probability, or both) during either the preparatory epoch or movement. If overlapping occurs and neurons respond to different parameters, the location is considered to be the multiparameter type (black dots).

Figure 5.

Interaction between reward–value and encoding of direction. A, Pie charts of neurons that are direction-sensitive only (Dir), reward-predicting only (Prb), or sensitive to both parameters (BP), during the preparatory and movement-related epochs (two-way ANOVA, probability × direction, p < 0.05). n refers to the number of responding neurons, and the percentages represent populations that encode each parameter according to responding neurons. * indicates that the ratio of GPi neurons that encode direction during movement execution is significantly larger than during preparatory activity or PUT neurons (χ² test, p < 0.001). B, The PUT and GPi population directional tuning curves were modulated by different reward probabilities [P(R) = 0.33, 0.67, and 1]. Each neuron selected in these populations encoded the direction with a cosinusoidal function (R² > 0.8; p < 0.05) and the value with a linear function (R² > 0.8; p < 0.05). The tuning curves were obtained with nonlinear regressions, with mean ± SEM parameters that generate a sinusoidal model for discharge rate. The curves for single-neuron analysis were aligned with their PD to plot populations. n refers to the number of neurons in each population that simultaneously encode both parameters.

Figure 6.

A, Polar diagrams illustrating the dual influence of movement direction and reward probability on the firing rate of two cells during target display (PD of 0°). B, Box plots showing PD related to three action values for neural populations that encode simultaneously both parameters (regression analysis, R² > 0.8; p < 0.05). For each reward probability, the PD was computed with nonlinear regression, and it was normalized according to the PD when P(R) = 1. The central line of the box plots represents the median, the edges of the box show the interquartile range, and the edges of the whiskers show the full extent of the overall distributions. n refers to the number of neurons in each population.

Figure 7.

A–C, Examples of three-dimensional representations of preparatory activity as a function of the PC and PNC. A, PUT neurons whose firing rate was modulated only according to PC. B, GPi neuron that was modulated only by PNC. C, GPi neuron that discharged according to probabilities related to both targets. Cues (PC and/or PNC) related to neural activity are shown in red. D, Distribution pie charts of neurons responding to cue values. The firing rate of some neurons was linearly modulated (regression, R² > 0.8; p < 0.05) by the chosen cue probability (C), the nonchosen cue probability (NC), or by both targets at the same time (BT). n refers to the number of responding neurons, and the percentages represent populations that encode each parameter according to responding neurons. * indicates that GPi neurons encode the probabilistic context significantly more frequently in the preparatory epoch than during movement (χ² test, p < 0.05).

Figure 8.

Example of GPi neuron that encodes simultaneously the reward value related to both targets (linear regression analysis, R² > 0.8; p < 0.05) and the location of the chosen cue (nonlinear regression analysis, R² > 0.8; p < 0.05) during the preparatory epoch. Each cosinusoidal curve was built for one target combination. In this example, the reward value of the nonchosen cue modulates the response for a chosen target with P(R) = 1 but not for P(R) = 0.67.