Different pedunculopontine tegmental neurons signal predicted and actual task rewards

Abstract

The dopamine system has been implicated in guiding behavior based on rewards. The pedunculopontine tegmental nucleus (PPTN) of the brainstem receives afferent inputs from reward-related structures, including the cerebral cortices and the basal ganglia, and in turn provides strong excitatory projections to dopamine neurons. This anatomical evidence predicts that PPTN neurons may carry reward information. To elucidate the functional role of the PPTN in reward-seeking behavior, we recorded single PPTN neurons while monkeys performed a visually guided saccade task in which the predicted reward value was informed by the shape of the fixation target. Two distinct groups of neurons, fixation target (FT) and reward delivery (RD) neurons, carried reward information. The activity of FT neurons persisted between FT onset and reward delivery, with the level of activity associated with the magnitude of the expected reward. RD neurons discharged phasically after reward delivery, with the levels of activity associated with the actual reward. These results suggest that separate populations of PPTN neurons signal predicted and actual reward values, both of which are necessary for the computation of reward prediction error as represented by dopamine neurons.

Figure 1.

Recording sites. Location of recording sites from MR images of monkey 1 (A) and monkey 2 (B). Photomicrograph of a histological section cut in the coronal plane, showing electrode tracks and the lesion marking the recording site in the PPTN for monkey 2 (C). IC, Inferior colliculus; SCP, superior cerebellar peduncle. Histological drawings are shown for monkey 1 (D) and monkey 2 (E) with an interval of ≈400 μm. Black dots and red circles indicate reconstructed recording sites and the histologically identified PPTN area, respectively.

Figure 2.

Diagrams for two-valued VGST and behavioral performance. A, Time diagram of the VGST. After fixation on the FT for 400–1000 ms, the FT disappeared after a 0 or 200 ms time gap and the ST was presented for 400–600 ms. Monkeys were required to make a saccade to the ST within 500 ms after the ST onset. Rewards for successful trials (RD) were delivered 100 ms after the ST offset. B, Diagrams of screen views. Arrows indicate directions of eye movement. The FT shape indicates reward value (square, three drops of juice; circle, one drop). C, RTft for success and error trials and RTst for success trials. Error bars indicate SEM. L and S indicate large- and small-reward trials, respectively. *p = 0.001, **p = 0.0001, and ***p = 0.05, t test.

Figure 3.

Responses of FT and RD neurons to task events. A, Rastergram for activity of a representative FT neuron during 10 successive normal cue trials, aligned to the FT onset. Red and green represent large- and small-reward trials, respectively. A, Blue squares and circles, The time of respective FT/cue onsets; black bars, ST onset; black triangles, saccade onset; blue bars, the times for RD, respectively. B, Peri-task-event SDF of the activity shown in A. Conventions for large- and small-reward trials are the same as for A. C, Population SDF for 30 FT neurons. Responses are aligned to the FT onset, ST onset, and RD. Population data were averaged for the FT neurons (n = 30) sampled in 1504 normal and 1123 reversed cue trials (squares and circles for large and small reward and vice versa), normalized for the peak response of the individual neurons. D–F, Similar rastergram and SDF for a representative RD neuron and population SDF for 15 RD neurons. Time axes in C and F are broken to align the responses to the onset of the FT, ST, and RD. Black bars indicate the periods of FTon, STon, and RD as denoted. Population SDF was averaged for 464 normal and 519 reverse cue trials.

Figure 4.

Information analyses of responses in FT and RD neurons. A, Pseudocolor plots of the instantaneous ROC values for the large and small rewards for activities in each of the 30 FT and 15 RD neurons. The plots are aligned to the FT and the ST onsets and RD. The horizontal white line separates the FT and RD. B, Cumulative plots of mutual information of the reward magnitude encoded by the 30 FT neurons (cyan traces) and 15 RD neurons (black traces). Time axes in A and B are broken to align the responses to the onset of FT, ST, and RD. C, Peri-RD histograms for the ROC value to return to the chance level (ROC <0.56) for the 30 FT neurons. D, Peri-RD histograms for the ROC value to exceed the chance level (ROC >0.56) for the 15 RD neurons. Data samples are the same as that for Figure 3, C and F.

Figure 5.

Response of FT and RD neurons to free and task rewards. A, B, Rastergram and SDF for representative and ensemble FT neurons (n = 19) aligned to delivery of free (red) and task (black) rewards. B, For 408 free reward and 1122 normal reward cue trials with a large reward, the responses represent the average firing frequency normalized for the peak responses of the individual neurons (19 FT neurons) whose number of trials was more than five for free reward with a large reward. C, D, Similar to A and B but for representative and ensemble response RD neurons (n = 9) sampled in 216 free reward and 459 normal cue trials with large rewards.

Figure 6.

Response of FT and RD neurons to reward omission and task reward. A, Rastergram for representative FT (red) and RD (black) neuronal activity aligned to reward omission for a normal cue trial with a large reward. B, Ensemble SDF for reward omission in 10 FT (red) and 6 RD (black) neurons from 97 and 52 normal cue trials, respectively. The responses represent the average firing frequency normalized for the peak responses of the individual neurons (10 FT and 6 RD neurons) whose number of trials was more than five for reward omission of a normal cue trial with a large reward. C, D, Responses to task reward for the same representative and ensemble FT and RD neurons as in A and B for 354 and 287 normal cue trials, respectively. Responses are aligned to the time of the FT presentation and the reward delivery (left and right dotted lines, respectively). Black horizontal bars in A and B indicate the range of the expected time of RD.

Figure 7.

Effects of cue reversal on the responses of FT and RD neurons. A, Responses of 22 FT neurons to an FT presentation (200–600 ms after FTon, FT/cue period) before and after reversal of the FT cue (“square-large and circle-small” to “square-small and circle-large”). The response represents the average firing frequency normalized for the peak responses of the individual neurons (22 FT neurons) whose number of trials was >10 for each of task attributes, including the cue shape and reward magnitude. Red and green lines connect the trials, in which the FT shapes were a square or circle, respectively. B, Similar to A but during the maintenance period of reward prediction (200–600 ms from FToff). C, Similar responses to A and B but for 11 RD neurons to RD (200–600 ms after RD, post-RD period). Blue and gray lines connect the large- and small-reward trials, respectively. D, RTft averaged for 2353 task trials, including 1144 normal (973 and 251 for monkeys 1 and 2, respectively) and 1209 reversed cues (890 and 239) in which the 22 FT and 11 RD neurons were sampled. The convention used to express the cue shape is the same as in A and B.

Figure 8.

Multiple linear regression analyses of FT and RD neuronal responses. A, The regression coefficients of task event variables (REW, red; FTS, green; RTft, purple; RTst, blue; DIR, black) during the three task periods [FT/cue period (FT), maintenance period of reward prediction (MP), and post-RD (RD)] for FT neurons. B, The same as in A but for RD neurons. Error bars are SEM across neurons (n = 22 in A and n = 11 in B). *p < 0.01, **p < 0.001, significance level of the regression coefficients.

Figure 9.

FT neuronal responses in failure and success trials of VGST tasks. A, Rastergram and SDF of a representative reward magnitude-dependent FT neuronal response for five successful, fixation hold error, and fixation error trials with normal cue. Black, The fixation error; cyan, fixation hold error; red, successful large-reward trials; green, successful small-reward trials. Bottom, Eye position in single representative case of the four trial categories using the same conventions. Upward arrow indicates the time of the fixation break. Two horizontal dotted lines indicate the fixation window within which the monkey was required to maintain the eye position. B, Population SDF of 30 reward-magnitude-dependent FT neurons averaged for 100 fixation error (black solid trace), 236 fixation hold error (solid green and red traces for large and small reward), and 2627 successful trials (dotted red and green dotted traces for large and small reward), aligned to task events including FT, ST, and RD, each of which contained 45, 131, and 1504 normal and 55, 105, and 1123 reversed cue trials, respectively. SDF is the population average normalized for the peaks of the mean individual neuronal responses for each category of success and failure trials, shown synchronized to the times of FT presentation, saccade onset, and RD (left, middle, and right dotted lines), respectively. C, Correlation coefficient (absolute value) plot of the FT neuronal responses shown in B with RTft (purple), RTst (blue), and reward magnitude (black). The horizontal dotted red line indicates the significance level (p = 0.05) of the correlations. D, Similar correlation plot to C but for four response categories, including those for trials with RTft <100 ms (cyan) and RTft ≥100 ms (green) and those for large and small rewards (solid and dotted traces). Number of trials for RTft <100 ms large- and small-reward and RTft ≥100 ms and RTft ≥100 ms large- and small-reward categories, 421, 325, 1005, and 776, respectively. E–G, Similar to B–D but for 52 reward-magnitude-independent FT responsive neurons (50 FT and 2 FT/RD neurons). The population SDF of the 52 reward-magnitude-independent neurons averaged for 115 fixation error, 325 fixation hold error, and 4470 successful trials, each of which contained 61, 174, and 2573 normal and 54, 151, and 1897 reverse cue trials, respectively. Number of trials for RTft <100 ms large- and small-reward and RTft ≥100 ms and RTft ≥100 ms large- and small-reward categories, 683, 747, 1275, and 1765, respectively.