Single-cell and population coding of expected reward probability in the orbitofrontal cortex of the rat

Abstract

The orbitofrontal cortex (OFC) has been implicated in decision-making under uncertainty, but it is unknown how information about the probability or uncertainty of future reward is coded by single orbitofrontal neurons and ensembles. We recorded neuronal ensembles in rat OFC during an olfactory discrimination task in which different odor stimuli predicted different reward probabilities. Single-unit firing patterns correlated to the expected reward probability primarily within an immobile waiting period before reward delivery but also when the rat executed movements toward the reward site. During these pre-reward periods, a subset of OFC neurons was sensitive to differences in probability but only very rarely discriminated on the basis of reward uncertainty. In the reward period, neurons responded during presentation or omission of reward or during both types of outcome. At the population level, neurons were characterized by a wide divergence in firing-rate variability attributable to expected probability. A population analysis using template matching as reconstruction method indicated that OFC generates a distributed representation of reward probability with a weak dependence on neuronal group size. The analysis furthermore confirmed that predictive information coded by OFC populations was quantitatively related to reward probability, but not to uncertainty.

Figure 1.

Localization of tetrode recording sites. As indicated by rectangles, recordings in all rats were localized in the ventral and lateral regions of the OFC, between 2.7 and 4.7 mm anterior from bregma. Recording depth ranged from approximately −3 to −5.5 mm (Paxinos and Watson, 2005). Several tetrode tracks are visible, as indicated by the black arrows. Black asterisks mark the lesion sites showing the final position of three tetrodes.

Figure 2.

Overview of behavioral correlates of neural activity changes observed during task performance. Perievent time histograms and raster plots showing examples of the observed task-related behavioral correlates. Examples from four different units recorded in four different sessions demonstrating correlates related to the following: A, odor sampling (synchronized on onset of odor presentation during p = 100% trials); B, movement activity preceding nose entry into the food trough (synchronized on entry of the food trough during p = 50% trials); C, waiting period of 1.5 s with nose in the food trough, synchronized on onset of waiting; and D, pellet delivery (both during p = 100% condition). These as well as the following histograms (Figs. 3, 4) are presented with a bin size of 100 ms. In all raster plots, individual consecutive trials are represented as horizontal lines, with the first trial at the top row. Horizontal calibration denotes time (seconds), and vertical calibration denotes firing rate (fr) (hertz).

Figure 3.

Differential firing in relation to reward probability. A, Example of a unit showing differential firing toward different reward probabilities during the waiting period. Activity is synchronized on nose entry into the food trough. Activity of this neuron decreases with decreasing reward probability: all four conditions differ significantly from each other except the p = 100 and 75% conditions. Fr, Firing rate. B, Overview of the significantly different firing rate profiles toward reward probability found during both the movement (dashed lines) and waiting (solid lines) periods. Different units are represented by different symbols. On the horizontal scale, reward probability (percentage) is plotted; the vertical scale displays the peak firing rate of individual units in association with different reward probabilities, normalized to the peak in the p = 100% condition. Note that only very few neurons exhibit distinct firing-rate peaks or valleys under maximal uncertainty (p = 50%). C, Activity of a neuron showing activity during the movement period as part of task performance (p = 75% condition) and during the same behavior in the ITI.

Figure 4.

Differential firing after reward delivery during the rewarded and unrewarded condition. Activity in rewarded trials is synchronized on pellet delivery, in unrewarded trials at the same time point as pellet delivery in rewarded trials. A, Example of a unit demonstrating an increase in firing activity solely during the rewarded trials. B, Activity of a different unit showing a double correlate: both during the reward and waiting period (starting at −1.5 s), this neuron showed a significant increase in firing activity. During the reward phase, the increase in firing rate during rewarded trials was significantly larger compared with unrewarded trials. No difference was found between the two conditions in the waiting period. C, This unit increased firing specifically during unrewarded trials during the task. Activity did not significantly change during the ITI.

Figure 5.

Distribution of population and parameter variability for the movement period (A, D), the waiting period (B, E), and the reward period (C, F). On average, parameter-related variability covered a broader range of values than population variability, spanning the whole range from 0.0 to 1.0 (average of 0.29, 0.22, and 0.27 for the movement, waiting, and reward periods, respectively). Variation in population variability was less, with values ranging from 0.5 to 1.0 (average of 0.76, 0.71, and 0.71 for the movement, waiting, and reward periods, respectively).

Figure 6.

Decoding of reward probability with an increasing number of neurons for the movement (A), waiting (B), and reward (C) periods. Decoding time windows used were 0.7, 1.5, and 5 s, respectively. The horizontal axis indicates the size of the reconstruction ensemble, and the vertical axis indicates the percentage of trials in which reward probability was correctly decoded. The horizontal dashed line indicates chance level (33.3%), and dotted lines flanking the curves represent the 95% confidence interval (2 times the SE of proportion).

Figure 7.

Decoding scores for the various reward probabilities with an increasing number of neurons during the movement period (A) and the waiting period (B) when spike vectors from unrewarded trials (p = 0%) were used for encoding. In both task phases, decoding for p = 50% trials was significantly above chance level (33.3%), whereas the score for p = 100% trials was below chance level. Decoding performance for p = 75% was either at chance level (movement period) or decreased below chance with increasing ensemble sizes (waiting period). When the decoding performance was averaged across all three probabilities, curves were at chance level for both trial phases (data not shown).