Prefrontal cortex activity related to abstract response strategies

Abstract

Many monkeys adopt abstract response strategies as they learn to map visual symbols to responses by trial and error. According to the repeat-stay strategy, if a symbol repeats from a previous, successful trial, the monkeys should stay with their most recent response choice. According to the change-shift strategy, if the symbol changes, the monkeys should shift to a different choice. We recorded the activity of prefrontal cortex neurons while monkeys chose responses according to these two strategies. Many neurons had activity selective for the strategy used. In a subsequent block of trials, the monkeys learned fixed stimulus-response mappings with the same stimuli. Some neurons had activity selective for choosing responses based on fixed mappings, others for choosing based on abstract strategies. These findings indicate that the prefrontal cortex contributes to the implementation of the abstract response strategies that monkeys use during trial-and-error learning.

Fig. 1

A. Sequence of task events. Gray rectangles represent the video screen, white squares show the three potential response targets (not to scale), the white dot illustrates the fixation point, and the converging dashed lines indicate gaze angle. Disappearance of the instruction stimulus was the trigger stimulus, after which the monkeys made a saccade (solid arrow) and maintained fixation at the chosen target. The target squares then filled with white, and reinforcement (dotted arrow), when appropriate. B. Penetration sites. Composite from both monkeys, relative to sulcal landmarks. C. Strategy task. Responses shown by the thick arrows in the middle column represent correct applications of the repeat-stay (pink background) or change-shift (blue background) strategies. + indicates a rewarded response; − an unrewarded response. If unrewarded, the monkeys then got a second chance to respond, and received reinforcement for choosing the saccade made least recently (right). D. Example sequence for the strategy task. The red circle and slash indicates a disallowed response.

Fig. 2

Performance curves. A, B. Strategy task. C, D. Mapping task. A, C. Monkey 1. B, D. Monkey 2. The percentage of correct responses, averaged over ^~130 problem sets, as a function of trial number. Blue curves show performance on repeat trials. Green curves show percentage of rewarded saccades, change trials only. Background shading indicates 95% confidence limits. Red curves show percentage of saccades that were chosen according to the change-shift strategy, change trials only.

Fig. 3

Two cells with strategy effects: A from the rostral part of PFdm, B from PFdl. The saccade directions are shown by the arrows. The squares on each line of the raster show the time that the trigger stimulus occurred; each dot corresponds to the time of a neuronal action potential. The background shading identifies the task periods. The cell in A had much greater activity for repeat trials (black) than for change trials (red) in the IS1 period, regardless of stimulus or saccade direction. The cell B had the opposite preference, and also showed some preference for responses to the left.

Fig. 4

Cell showing that the strategy effect does not simply reflect detection of whether the IS repeats from trial-to-trial. Format as in Fig. 3. A. Change trials. B. Repeat trials. Note selectivity for change trials, but only for upward (top row) and (to a lesser extent) leftward responses (middle row).

Fig. 5

ROC plots. Colors show the area under the ROC curve for each individual cell, ranked according to the time at which this signal develops after stimulus onset. A. All neurons in Monkey 2. B. Neurons with an ROC value >0.6 for 4 consecutive bins and a preference for the repeat-stay strategy. C. As in B, but with a preference for the change-shift strategy. D and E. Data from C and B, respectively, with a color scale approximating that used by Wallis and Miller (2003b). F and G. ROC plots from Monkey 1 in the format of D and E.

Fig. 6

Cell preferring the change-shift strategy and lacking a major influence of reward prediction. From PFdl. A. Standard version of the strategy task, with correct change-shift choices rewarded at a 50% rate. B. High-reward version of the task, using the same stimulus set, with correct change-shift choices rewarded at a 90% rate to more closely match the reward rate for repeat trials. C. Comparison of the strategy score and the reward-prediction score. Percent of cells with activity better matching reward probability (blue) or strategy (magenta). Abbreviations: CSh, change-shift; IS1, early instruction-stimulus period; IS2, late instruction-stimulus period; rew, reward; RMT, reaction- and movement-time period.

Fig. 7

Cell preferring the mapping task. This neuron was located in PFdm (see Fig. 1B). A, B and C each show neuronal activity relative to the onset of the instruction stimulus. Neuronal activity averages: red for the mapping task, black for the strategy task. Change trials differ (p<0.05, Mann-Whitney U Test) but repeat trials do not (p=0.49). D. Three ISs, with arrows indicating the correct action for each. E. Percent of cells by task period showing a task effect, for each monkey. Abbreviation: rew, reward.

Supplemental Figure 1

The results from Suppl. Table 5 as a bar graph, both monkeys combined. Abbreviations: IS, instruction stimulus; Resp, response; RMT, reaction- and movement-time period. Reward-prediction score versus strategy score To further evaluate the extent to which the cell’s activity reflected reward prediction or anticipation, as opposed to the repeat-stay and change-shift strategies, we calculated two scores, which compared each cell’s activity across several tasks: a reward-prediction score and a strategy score. In each calculation, all activity was normalized by the maximal activity for that cell and task period. Thus, normalized activity ranged between 0 and 1, and the activity for several trial types (repeat trials, change trials, mapping trials, and second-chance trials) contributed a value in that range. For the reward-prediction score, a model assumed either a strong correlation of activity with approximate reward rate or a strong anti-correlation. Low reward rates were those of approximately 50%, high ones were those of approximately 90%. High-reward situations were second-chance trials in the standard version of the strategy task, trials late in the mapping task, and either repeat or change trials in the high-reward version of the strategy task. Low-reward situations included those early in the mapping task for change trials and all change trials in the standard version of the strategy task. A score near 0.5 indicated that the cell’s activity did not correspond to the prospect for reward, and the extent of deviation of that score towards 0 or 1 indicated a progressively larger degree of correspondence to the reward-prediction model (either in correlation or in anti-correlation with the probability of reward). Analogously, for the strategy score, a different model assumed that the change-shift strategy would be associated with high activity levels on change trials and low activity on the repeat trials. For this model, too, a score of 0.5 indicated that the cell’s activity did not accord with either the repeat-stay strategy or the change-shift strategy, and the extent of deviation of that score towards 0 or 1 indicated a progressively larger degree of correspondence to the strategy model (either in correlation and anti-correlation with the expectation for change-shift preferences, the anti-correlation reflecting repeat-stay preferences). To identify a preference for the strategy scores versus the reward-prediction score, we compared the absolute value of the difference between both scores and 0.5. Fig. 6C and Suppl. Fig. 2 (below) shows that the strategy score predominated over the reward-prediction score. This finding was consistent in both monkeys and across task periods, with the exception of the post-reward period, in which the number of cells with preferences for reward prediction decreased dramatically (Fig. 6C). ANOVA (α=0.05) revealed no significant effects of Monkey for either the strategy score or the reward-prediction score (F_1,1=2.2, p=0.14 for the strategy score; F_1,1=1.2, p=0.28 for the reward-prediction score). Accordingly, data from both monkeys were combined in Suppl. Fig. 2.

Supplemental Figure 2

Scatter plots of strategy score and reward-prediction score. Deviation from 0.5, which represents the worst fit to each model. Abbreviations: IS1, early instruction-stimulus period; IS2, late instruction-stimulus period; RMT, reaction- and movement-time period; Pre-rew, pre-reward period; Post-rew, post-reward period.

Supplemental Figure 3

Distribution of I_task across the population of cells for each period. A, B. Frequency distributions for the task-effect index (I_task), for each of three task periods, labeled at bottom. C, D. Frequency distributions for strategy-effect index (I_strat). Abbreviations: IS1, early instruction-stimulus period; IS2, late instruction-stimulus period; RMT, reaction- and movement-time period. Note that the neuronal subpopulation can differ across periods.

Supplemental Figure 4

Continuation of Suppl. Fig. 3 for pre-reward and post-reward periods. The right plot shows the relative proportion of cases with a preference for the change-shift strategy (CSh, gray bars) and the repeat-stay strategy (RSt, white bars).

Supplemental Figure 5

Surface plots of the locations of cells with strategy effects, by task period. A. Monkey 1. B. Monkey 2. Inset: composite of both monkeys, all task periods. Shading shows penetrations with either more than 30% of cells showing a significant preference for the strategy task (black circles), with some of that type, but less than 30% (gray circles), or no such cells (unfilled circles). Rostral to the right; dorsal up.

Supplemental Figure 6

Surface plots of the locations of cells with task effects, by task period. A. Monkey 1. B. Monkey 2. Inset: composite of both monkeys, all task periods. Format as in Suppl. Fig. 5.