Theta-band phase locking of orbitofrontal neurons during reward expectancy

Abstract

The expectancy of a rewarding outcome following actions and cues is coded by a network of brain structures including the orbitofrontal cortex. Thus far, predicted reward was considered to be coded by time-averaged spike rates of neurons. However, besides firing rate, the precise timing of action potentials in relation to ongoing oscillations in local field potentials is thought to be of importance for effective communication between brain areas. We performed multineuron and field potential recordings in orbitofrontal cortex of rats performing olfactory discrimination learning to study the temporal structure of coding predictive of outcome. After associative learning, field potentials were marked by theta oscillations, both in advance and during delivery of reward. Orbitofrontal neurons, especially those coding information about upcoming reward with their firing rate, phase locked to these oscillations in anticipation of reward. When established associations were reversed, phase locking collapsed in the anticipatory task phase, but returned when reward became predictable again after relearning. Behaviorally, the outcome anticipation phase was marked by licking responses, but the frequency of lick responses was dissociated from the strength of theta-band phase locking. The strength of theta-band phase locking by orbitofrontal neurons robustly follows the dynamics of associative learning as measured by behavior and correlates with the rat's current outcome expectancy. Theta-band phase locking may facilitate communication of outcome-related information between reward-related brain areas and offers a novel mechanism for coding value signals during reinforcement learning.

Figure 1.

Task-related theta oscillations and phase locking. A, Operant chamber; impression of rat making odor poke (left) and fluid poke (right). B, Raw LFP segments recorded during different task periods (black) and bandpass-filtered LFP (4–12 Hz) (green trace, hit trial; red, false alarm trial) and spike trains from one unit, recorded on a different tetrode (vertical green/red bars). psd, prestimulus delay. C, LFP power modulation, synchronized in time on sucrose delivery. Pseudocolors indicate averaged percentage increase in LFP power compared to the intertrial interval period. Green dashed line, Median time of odor onset; red dashed line, median time of odor offset; white dashed line, start of waiting period; white solid line, sucrose delivery. Theta oscillations (9 Hz) found during odor sampling are not visible here due to synchronization on reward delivery. D, Spike–field phase-locking spectrum averaged across all task-segments and all units. Shaded regions indicate 95% bootstrapped confidence intervals on the mean. E, Polar histogram of mean phase values for all cells. Shaded part corresponds to number of significantly phase-locked units (270 of 525, Rayleigh test, p < 0.05). Mean phase, 11.3°. Distribution of phases deviated significantly from uniform distribution (p < 0.001).

Figure 2.

Modulation of spike–field phase locking during reward expectation and delivery. A, Time course of average (across all cells) phase locking centered on reward delivery. Phase locking was calculated in 500 ms symmetric windows centered on each time point. Solid line, Reward delivery; dashed line, start of waiting period. B, Average spike–field phase-locking spectra during waiting period, separated for sucrose, quinine delivery, and during (unrewarded) intertrial interval fluid pokes. Shaded regions indicate 95% bootstrapped confidence intervals on the mean. C, Same as in B, but now during fluid delivery period. D, Bar plot of phase-locking values at 6 Hz (mean ± SEM) for different task periods. M, Movement; D_S, sucrose delivery; D_Q, quinine delivery; W_S, waiting period before sucrose; W_Q, waiting period before quinine. Numbers indicate percentages of cells significantly phase locked (Rayleigh's test p < 0.05). Comparisons, ***p < 0.001; **p < 0.01; *p < 0.05, Wilcoxon's matched-pairs signed-rank test.

Figure 3.

Learning-related changes in phase locking and LFP. A, Spike–field phase-locking spectra for prereversal phase, separated in early (cyan) and late (magenta) trials. B, Same as A, but now for reversal phase. C, Trial-by-trial modulation of theta power (6 Hz) relative to reversal. Pseudocolor scale indicates theta power modulation compared to baseline, averaged over contributing sessions (see Materials and Methods). Before reversal, odor 1 predicts sucrose (O₁-S). After reversal, odor 1 predicts quinine (O₁-Q). Dashed white line, Start of waiting period; solid white line, fluid delivery; ACQ, acquisition; REV, reversal. D, Same as C, but now synchronized on fluid delivery following odor 2. E, Hit/false alarm trials show opposite patterns for theta power COM (see Materials and Methods). Dashed lines indicate moment of fluid delivery. Red and green lines indicate mean shifts. F, Trial-by-trial timing of theta onset on hit trials in relation to reversal. Green line indicates mean first time point in time window where instant theta power reached >50% of maximum theta power on that trial. Shading indicates SD.

Figure 4.

Relationship between behavioral firing-rate correlates and spike–field phase locking for different trial periods. A, Regression coefficients for the four categories of possible behavioral correlates. Only the waiting period correlate is a significant positive predictor of phase-locking values during the waiting period (tested at 6 Hz). Shaded regions indicate SEM. B, As in A, but now for the fluid delivery period. C, Phase-locking spectra separate for trials in which the outcome decoded from the neuron's firing rate matched (green) or failed to match (red) the actual outcome. Shaded regions indicate 95% bootstrapped confidence intervals on the mean. There was a significant difference at 6 Hz (p < 0.01, Wilcoxon's matched-pairs signed-rank test).