Contrasting Roles for Orbitofrontal Cortex and Amygdala in Credit Assignment and Learning in Macaques

Abstract

Recent studies have challenged the view that orbitofrontal cortex (OFC) and amygdala mediate flexible reward-guided behavior. We trained macaques to perform an object discrimination reversal task during fMRI sessions and identified a lateral OFC (lOFC) region in which activity predicted adaptive win-stay/lose-shift behavior. Amygdala and lOFC activity was more strongly coupled on lose-shift trials. However, lOFC-amygdala coupling was also modulated by the relevance of reward information in a manner consistent with a role in establishing how credit for reward should be assigned. Day-to-day fluctuations in signals and signal coupling were correlated with day-to-day fluctuation in performance. A second experiment confirmed the existence of signals for adaptive stay/shift behavior in lOFC and reflecting irrelevant reward in the amygdala in a probabilistic learning task. Our data demonstrate that OFC and amygdala each make unique contributions to flexible behavior and credit assignment.

Figure 1

Object Discrimination Reversal Task (A) Each trial started with an inter-trial interval (ITI) showing a blank screen. Two options were then presented on the screen, monkeys chose one of the options by reaching the touch sensor placed in front of it (decision phase). Juice reward was delivered if a correct option was chosen (outcome phase). (B) The task was designed with a two-option deterministic reversal schedule. Each session began with one correct option that led to a reward and one incorrect option that did not lead to a reward. The stimulus-reward contingencies reversed after monkeys performed 50 and 100 rewarded trials. (C) On average, the accuracies of all monkeys were low on the early trials in a block and gradually increased. (D) Performance averaged across testing sessions within subject. Each line represents data from one subject. The raster plots in (C) and (D) indicate trials in a block with accuracies significantly higher than 0.5 (p < 0.05). (E and F) Example sessions from two different subjects. Accuracies were calculated by using a moving average window of 5 trials. The dotted lines indicate reversals. The raster plots indicate the reward (green) and no reward (red) outcome events.

Figure 2

Win-Stay/Lose-Shift Signal in the lOFC (A) A whole-brain analysis showing a signal in the lOFC that was related to the occurrence of an outcome event. (B) A whole-brain analysis showing a signal in the lOFC that predicted win-stay/lose-shift behavior. (C) lOFC (16, 8, −4; green) BOLD activity was extracted for ROI analysis. (D and E) BOLD signal time course in the lOFC from an example session. The task events of win-stay, win-shift, lose-stay, and lose-shift are labeled in green, orange, red, and blue, respectively. (F) The BOLD signal was time locked at the outcome phase of the task and averaged across testing sessions and subjects. (G) The lOFC showed WSLS activity that ramped up after the onset of the outcome phase and peaked at around 4 s (green). (H) All four subjects consistently showed a WSLS signal after the outcome was revealed at 0 s. (I) The signal was extracted from the time window indicated by the bracket above the time course (which corresponds to the full-width half-maximum of the peak established using a leave-one-out procedure) and correlated with behavior. Testing sessions with larger WSLS signals in the lOFC were related to higher accuracies during the learning phase (first 9 trials in a block). (J) The sizes of the WSLS signal had no relationship with accuracies in the post-learning phase (after 30 trials in a block). Each type of marker symbol in (I) and (J) represents data from one animal.

Figure 3

Lose-Shift Signal in the Amygdala (A) The amygdala ROI (14, −3, −13; red) in sagittal view (top panel) and coronal view (bottom panel) for BOLD activity extraction. (B) There was no clear WSLS (green) signal in the amygdala.

Figure 4

Distinctive Features of lOFC and Amygdala WSLS Signals First, we split the WSLS signals by whether a trial occurred in a period when accuracy was below 0.7. The amygdala did not encode a WSLS signal when monkeys’ accuracies were high (A) but did so when accuracies were low (B). In contrast, the lOFC encoded WSLS signals no matter whether accuracies were high or low (C and D). Second, we split the WSLS signal by whether it was a win or lose trial. In other words, win-stay and lose-shift signals were investigated separately. The amygdala did not have a win-stay signal (E), but three monkeys encoded a positive lose-shift signal (F). In contrast, both win-stay and lose-shift signals were seen in the lOFC (G and H). In the amygdala, the lose-shift signal was strongest, and statistically significant, when focusing on trials with low accuracies (I and J).

Figure 5

PPI between lOFC and Amygdala that Guided Behavioral Change (A) lOFC and amygdala exhibited stronger connectivity when monkeys performed lose-shift rather than win-stay behavior. (B) Larger PPI effect sizes were related to higher proportions of lose-shift behaviors. Each type of marker symbol in (B) represents data from one animal.

Figure 6

Previous Reward Signals in the lOFC, ACC, and Amygdala (A) The lOFC did not carry signals related to whether or not a reward had been delivered on the previous trial at the time of the outcome phase on the subsequent trial. (B) The amygdala encoded the reward of the previous trial at the time of the decision and outcome phases of the current trial.

Figure 7

PPI between lOFC and Amygdala that Avoided Irrelevant Reward Information (A) The lOFC-amygdala connectivity was negatively modulated as a function of the previous reward. (B–D) Testing sessions with stronger negative modulation was marginally related to more stay decisions after a win trial when all trials were considered together (B) and statistically significantly related to the presence of more stay decisions when analysis was focused on the first five consecutive win trials (C). There was no relationship between the same neural signal and behavior after five consecutive win trials (D). Each type of marker symbol in (B)–(D) represents data from one animal.

Figure 8

Probabilistic Learning Task: Experiment 2 (A) On every trial, two out of three options were offered to the animals to choose. (B) Each option was associated with a probability of reward, as opposed to being linked in a deterministic manner as in the ODR task in Experiment 1. Instead of relying on the outcome of the previous decision and choosing according to a WSLS strategy, animals had to integrate the reward history of an option over an extended number of trials to make adaptive choices. (C) When the value of the chosen option was larger than that of the unchosen option (adaptive), animals should stay with the same choice when the same pair of options was offered on the next trial; however, animals should shift to the unchosen option when the value of the chosen option was smaller than the unchosen option (maladaptive). In other words, animals should follow an adaptive-stay/maladaptive-shift (ASMS) strategy and use of just such a strategy was associated with lOFC (16, 8, −4) activity. (D) In contrast, the amygdala only showed a marginally significant ASMS signal. (E and F) As in the ODR task in Experiment 1, a signal related to whether or not reward was delivered on the previous trial was absent in the lOFC (E) but present in the amygdala (F).