Adaptation of prefrontal cortical firing patterns and their fidelity to changes in action-reward contingencies

Abstract

Animals adapt action-selection policies when the relationship between possible actions and associated outcomes changes. Prefrontal cortical neurons vary their discharge patterns depending on action choice and rewards received and undoubtedly play a pivotal role in maintaining and adapting action policies. Here, we recorded neurons from the medial precentral subregion of mouse prefrontal cortex to examine neural substrates of goal-directed behavior. Discharge patterns were recorded after animals developed stable action-selection policies, wherein four possible action sequences were invariably related to different reward magnitudes and during adaptation to changes in the action-reward contingencies. During the adaptation period, when the same action sequence resulted in different reward magnitudes, many neurons (38%) exhibited significantly different discharge patterns for identical action sequences, well before reaching the reward site. In addition, trial-to-trial reliability of ensemble pattern production leading up to reward was found to vary both positively and negatively with increases and decreases in reward magnitude, respectively. Pairwise analyses of simultaneously recorded neurons revealed that decreased reliability in part reflected fluctuations between different ensemble activity patterns as opposed to within-pattern variability. Increases in reliability were related to an increased probability of both selecting highly rewarding actions and completing such actions without pause or reversal, whereas decreases in reliability were associated with the opposite pattern. Thus, we suggest that both the spatiotemporal pattern and fidelity of prefrontal cortical discharge are impacted by action-outcome relationships and that each of these features serve to adapt action choices and maintain behaviors leading to reward.

Figure 1.

Experimental design and behavioral performance. A, Schematic of the maze apparatus used. Under conditions of dim light on a 60 × 60 cm black platform, movement of mice was confined to corridors (shown here in white) defined by black walls 5 cm in height. On a given trial, the beginning of which was defined by crossing of the start line (S), the animal made two left/right turn choices. The first of these was always at turn A and the second at either turn B or C. Depending on the sequence of turns taken, the animal arrived at one of four goal sites in which NR, an SR (1 chocolate pellet), or a HR (3 chocolate pellets) was obtained. Animals returned to the start site via a separate path (gray broken line). B, The probability of choosing the L–R, R–L, L–L, and R–R sequence of turn choices is plotted as a function of trial blocks across days of training and recording. All mice developed a bias toward the action sequence (R–R) that led to the HR goal site (D1–D7) but still occasionally selected the L–R action sequence leading to the SR goal site. These biases were retained after implantation of recording wires (D13–D17) but changed to match an altered pattern of reward structure wherein the L–L action sequence was associated with large reward, the R–L sequence was associated with small reward, and the L–R and R–R sequences were associated with no reward.

Figure 2.

Localization of recording sites and normalization of firing rate data. A, Analysis of Nissl-stained coronal sections identified marker lesions made at the site of single-unit recordings (black arrow marks the deepest penetration of the stereotrode array). All recordings were made within the medial precentral region of the prefrontal cortex (filled black circle represents the full area of recording sites across all animals). B, The animal's position between the start and goal sites was normalized to account for slight lap-to-lap differences in trajectory and in the time elapsed during traversal of individual path epochs (for details, see Materials and Methods and Results). On a lap-to-lap basis, there was variation in time between crossings of lines defining arrival at start, turn, and goal sites [e.g., the larger time taken between turn B and the goal site (E) of lap 3 vs laps 1 and 2]. Here, this variation is exaggerated to emphasize the alignment of crossing times produced by the normalization process (middle). Alignment permitted calculation of mean firing rates and their associated variance across multiple laps despite minor differences in the timing and trajectory of path traversals (bottom). M1, Primary motor cortex; AI, primary auditory cortex; Cg1, area 1 of the cingulate cortex; PrL, prelimbic cortex; IL, infralimbic cortex.

Figure 3.

Comparison of MPC discharge patterns during HR and SR path traversals. A. Color-mapped firing rates of 93 MPC neurons during uninterrupted traversals across the HR (top image) and SR (bottom image) paths (dark blue to deep red are 0 to 100% of peak discharge rate; path positions were normalized and linearized by the process depicted in Fig. 2B). As for black lines in C and D, white lines represent where animals crossed lines defining start (S), goal (E), and turn (A, B) sites. The top-to-bottom ordering of neurons was determined by the positioning of peak discharge rates observed on the HR path. The smooth tiling of peak discharge points indicates that MPC ensembles represent all points of path traversals. The same ordering is used for the SR color map to highlight differences in firing rate for HR and SR path. B, Firing rate as a function of path position for two MPC neurons. Black and gray traces correspond, respectively, to SR and HR path traversals. C, Significant differences in discharge rate for 93 individual neurons were determined for corresponding positions of the HR and SR paths. For each neuron, path positions associated with significant firing rate differences are depicted as gray dots. For any given position (x-axis), the percentage of all neurons exhibiting significant discharge increases or decreases was determined and is given by the blue line. The mean percentage of neurons expected to exhibit significant differences by chance and the 95% confidence interval for this mean were determined by randomizing individual HR and SR trials (see Materials and Methods) and are given, respectively, by the full and dotted red lines. D, Left, Mean, peak-normalized firing rates for 93 neurons across the HR and SR paths. Overall firing rates were very similar across all stages of task performance. Right, Movement velocity (full lines, left y-axis) and variability in movement velocity (broken lines, right y-axis) did not differ between HR and SR path traversals.

Figure 4.

Adaptations in discharge patterns of MPC neurons across the same action sequence are driven by learning of novel action sequence–reward amount pairings. A, Color maps of peak-normalized discharge rates (dark blue to deep red are 0 to 100%) across the same path for the early and late adaptation periods. Top-to-bottom ordering of neurons is based on the position of peak discharge occurring during the late adaptation period. The same ordering is used for the early adaptation period to emphasize differences in discharge pattern associated with the development of a bias toward selection of this path. B, Firing rate as a function of path position for two MPC neurons. Black and gray traces correspond, respectively, to late- and early-stage traversals of a path newly associated with high reward. C, For each cell, depicted by gray dots are the position(s) along the new HR path in which discharge rates changed between traversals made during the early and late adaptation periods. For all positions along the path (x-axis), the blue trace reflects the percentage of all neurons exhibiting rate changes. The mean percentage of neurons expected to exhibit significant differences by chance was determined by randomizing individual early and late trials (see Materials and Methods) and is given by the red line (the 95% confidence interval for this mean is given by the dotted red line). D, Changes in reward amount did not alter overall MPC firing rates (left). Firing rate adaptations driven by changes in reward amount could not be explained by altered movement velocity or variability in movement velocity (right plot, full lines represent movement velocity given by the left y-axis, broken lines correspond to variability in movement velocity given by the right y-axis).

Figure 5.

Relationship between discharge rate adaptations and the probability of path selection. A, The mean positional firing rates of a neuron during early (gray) and late (black) traversals of the new HR path. The amplitudes of activity increases before both the start and reward lines significantly increased over the course of the adaptation period. B, Black dots represent the peak firing rate (right y-axis) of the pre-start activity increase calculated in partially overlapping seven-trial blocks (e.g., for trials 1–7, 4–10, 7–13). The correlation between trial block number and firing rate was significant (r² = 0.73; p < 0.001). Gray squares represent the probability (left y-axis) of selecting the new HR sequence. Selection probability correlated significantly with trial block number (r² = 0.62; p < 0.005).

Figure 6.

Coefficients of firing rate variance for MPC discharge patterns are related to reward amount. A, Plot of mean firing rate versus cross-lap firing rate variance for 93 MPC neurons. B, 1/CV for the full population of MPC neurons recorded during traversals of HR and SR paths. As measured by the reciprocal of the cross-lap CV of discharge, the reliability with which MPC activity patterns were produced was greater for the path associated with high reward (black bars indicate path positions for which significant differences were found; p < 0.05). Similar results were obtained when HR traversals were randomly subsampled to match the number of SR traversals for each recording (dotted black line; see Materials and Methods). C, Mean 1/CV for HR path traversals was significantly increased over that for SR path traversals. The opposite pattern was found for mean number of navigational errors on HR versus SR paths. D, 1/CV for early (gray lines) versus late (black lines) adaptation periods as a function of position along the new HR path (full lines) and a new no-reward path (dashed lines, formerly the HR path). Horizontal gray bars along the top mark significant differences in early versus late CVs for the new no-reward path. Black bars mark the same for the new HR path. E, Across the entirety of the new NR path, mean ± SD 1/CV values decreased significantly with experience while, at the same time, mean 1/CV values increased for the new HR path (n = 78; *p < 0.001). e, Early; l, late.

Figure 7.

Noise correlations between neuron pairs differ across HR and SR paths. A, Plot of firing rates for two simultaneously recorded neurons (dark blue and red traces of the top and middle graphs, respectively) across 10 different traversals of the SR path. Positions 1–40 for each traversal are laid end to end. Dotted vertical lines mark the end of each traversal. Light blue and pink traces in each represent the mean firing rate values for each position bin across all 10 traversals. The mean pattern is repeated 10 times for comparison with the actual firing rate observed on each trial. The positive signal correlation for these two neurons, given to the right, reflects the similarity of their mean discharge patterns across this path. The bottom graph depicts, for each traversal, the deviation of the firing rate of each neuron from the mean firing rate for that position. The correlation between values (the noise correlation) for the two neurons is given to the right. The correlation of deviations from mean firing rate for these two neurons was also significantly positive. B, Data are presented in the same way as A but for a pair of neurons bearing negative signal and noise correlations. C, Plotted are signal (or “rate”) and noise (“dev.”) correlations for all neuron pairs for HR (left) and SR (right) paths. A strong correlation between these correlation values was obtained for both paths. D, Mean ± SD absolute values of pairwise correlations of rate (i.e., signal correlations) and rate deviations (i.e., noise correlations) for all simultaneously recorded neuron pairs (n = 222; gray bars, HR path traversals; black bars, SR path traversals). Mean noise correlations were significantly higher for SR path traversals (*p < 0.001).