Lateral orbitofrontal cortex anticipates choices and integrates prior with current information

Abstract

Adaptive behavior requires integrating prior with current information to anticipate upcoming events. Brain structures related to this computation should bring relevant signals from the recent past into the present. Here we report that rats can integrate the most recent prior information with sensory information, thereby improving behavior on a perceptual decision-making task with outcome-dependent past trial history. We find that anticipatory signals in the orbitofrontal cortex about upcoming choice increase over time and are even present before stimulus onset. These neuronal signals also represent the stimulus and relevant second-order combinations of past state variables. The encoding of choice, stimulus and second-order past state variables resides, up to movement onset, in overlapping populations. The neuronal representation of choice before stimulus onset and its build-up once the stimulus is presented suggest that orbitofrontal cortex plays a role in transforming immediate prior and stimulus information into choices using a compact state-space representation.

Figure 1. Rats use the trial-by-trial-dependent contingencies of the task to improve their performance.

(a) Schematic of the task (see Methods for details). Two identical, consecutive tones (T1 and T2) are presented to the rats (top panel). Inter-tone intervals (ITIs) can belong to two stimulus categories: short, S=s, or long, S=l. Each category has four possible ITIs (short: 50, 100, 150 and 200 ms; long: 350, 400, 450, 500 ms). The vertical dotted line represents the decision boundary at 275 ms. Difficult ITIs, depicted in gray, lie close to the decision boundary. Sequence of events within a trial (bottom panel): from trial initiation to choice. Rats self-initiate the trial and sample the stimuli in the central socket. They are rewarded with water if they poke the right socket when the stimulus is long, and the left socket when the stimulus is short (for rat 3 the contingency was the opposite). (b) The sequence of trials follows an outcome-coupled hidden Markov chain (see also Supplementary Fig. 1): a new random stimulus condition is presented after a correct response (+), while the same stimulus condition is presented after an incorrect response (−). (c) Psychometric curves (probability choice long versus ITI) after correct responses (green line) and after incorrect responses (red line) for an example rat (left panel) and for all rats (right). The slope of the psychometric curves after incorrect responses substantially and significantly increases relative to the slope of the curve after a correct response. Error bars (shaded) are estimated by bootstrap (one s.d.). (d) Probability of lose-switch versus probability of win-stay. Each point corresponds to a different session. Rats predominantly follow a lose-switch over a win-stay strategy. No strategy being followed corresponds to the point (0.5, 0.5) in the plot.

Figure 2. OFC neurons encode relevant past information and anticipate upcoming choices even before stimulus onset.

(a) Electrode's path (dashed red line) and recording sites (solid red line) in rat lOFC depicted in a coronal section representation at 3.7 mm AP, 2.5 mm ML and 1.6 mm DV from Bregma. (b) Neuronal responses were aligned to trial initiation, defined as the time at which the rat starts the trial by poking the central socket. (c) Example neuron encoding reward in the previous trial R₋₁ (either + or −). This particular neuron fires more strongly for non-rewarded previous trials. (d) Example neuron representing choice in the previous trial C₋₁ (either s or l). (e) Example neuron tracking second-order prior X₋₁=C₋₁ × R₋₁ (either s or l). (f) Example neuron encoding upcoming choice C₀ as a function of time. This neuron conveys information about rat's upcoming choice before stimulus onset (stimulus onset always happens to the right of the shaded area). (c–f): Time zero corresponds to trial initiation. The period of time between trial initiation and the shorter stimulus onset (150 ms) is indicated with shaded areas. Curves correspond to trial-averaged firing rates smoothed with a causal sliding rectangular window (size of 100 ms and step of 50 ms), and shaded areas around them correspond to s.e.m. Insets represent spike waveform for each neuron (black line, mean; shaded area, s.d.).

Figure 3. OFC neurons encode essential quantities throughout the trial.

(a–d) Neuronal responses were aligned to stimulus offset (a). The firing rate of OFC neurons was modulated in a time period before stimulus offset (150 ms, shaded areas in b–d) by the stimulus (b), the upcoming choice (c) and the expected value of the outcome (d). (e–h) Neuronal responses were aligned to lateral nose poking onset, choice period (150 ms) (e). The firing rate of OFC neurons represented the stimulus (f), the current choice (g) and the outcome (h). In the two periods, signals about upcoming and current choice were very conspicuous. Time zero corresponds to stimulus offset (b–d) and lateral nose poking onset (f–h). Curves correspond to trial-averaged firing rates smoothed with a causal sliding rectangular window (size of 100 ms and step of 50 ms), and shaded areas around them correspond to s.e.m. Insets represent spike waveform for each neuron (black line, mean; shaded area, s.d.).

Figure 4. Neurons in lOFC integrate prior with current sensory information and encode upcoming choice.

(a) Fraction of neurons with significant regressors (see Methods) for each of the variables listed in the horizontal axis. Upcoming choice C₀, previous choice C₋₁, upcoming stimulus S₀, second-order prior X₋₁, upcoming outcome R₀ and previous outcome R₋₁ are significantly encoded in the population. Upcoming choice C₀ is encoded by lOFC neurons even before stimulus is presented. Variables that extend further back into the past are not significantly encoded in lOFC. (b–c) Fractions of neurons with significant regressors during a time period before stimulus offset (b), and during a time period after lateral nose poking (c). (d) Fractions of neurons encoding upcoming choice C₀, stimulus S₀, second-order prior X₋₁, previous choice C₋₁ and previous reward R₋₁ at trial initiation (pre-stimulus), stimulus offset and choice periods. Note that larger fractions of neurons have choice-related signals as time progresses through the trial. (a–c) One-tailed binomial test, *=P<0.05, **=P<0.01, ***=P<0.001. Shaded rectangle corresponds to non-significant fraction of neurons (P>0.05).

Figure 5. Encoding of essential variables for the task is stable before motor execution of the choice.

(a) Correlation coefficient between weights estimated at trial initiation and before stimulus offset for several variables (see Methods). Upcoming choice C₀, second-order prior X₋₁, previous choice C₋₁ and previous reward R₋₁ are stably encoded in lOFC. (b) None of the correlation coefficients between weights computed just before stimulus offset and during the choice period were significantly different from zero. (c) There is a positive correlation between the encoding weights associated with second-order prior and upcoming choices at the pre-stimulus period. Two-tailed permutation test (see Methods), *=P<0.05, **=P<0.01, ***=P<0.001.

Figure 6. Population decoding reveals pre-stimulus neural representations of second-order prior and upcoming choice.

(a) Decoding performance for upcoming choice at fixed second-order prior increases with the number of neurons in the ensemble (one to three) across all ensembles (gray), and does so more strongly for the 10% most informative ensembles (orange). Only trials after correct responses are used for the analysis. Left panel: schematic showing that the pre-stimulus firing rate of neuron i in the ensemble can possibly depend at most on second-order prior X₋₁ and upcoming choice C₀, as previously revealed by a linear analysis (Supplementary Fig. 4). To show that upcoming choice truly modulates neural activity, we performed a conditioned analysis by which the value of the second-order prior is fixed (gray-blue) while a linear classifier is trained to predict upcoming choice from the activity patterns in OFC (see Methods). (b) Decoding performance for second-order prior at fixed upcoming choice. Colour code and analysis are as in the previous panel. One-tailed permutation test, *=P<0.05, **=P<0.01, ***=P<0.001.

Figure 7. Population decoding analysis reveals a hierarchy of encoded variables.

(a) Decoding performance for each quantity at the pre-stimulus period as a function of the number of neurons in the ensemble (one to three) across all ensembles (gray) and for the 10% most informative ensembles (orange). All trials are used for the analysis. (b,c) Same as in the previous panel for stimulus offset and choice periods. This analysis reveals a hierarchy of encoding, with upcoming choice C₀ and second-order prior X₋₁ being two of the most strongly encoded variables. One-tailed permutation test, *=P<0.05, **=P<0.01, ***=P<0.001.