Differential coding of uncertain reward in rat insular and orbitofrontal cortex

Abstract

Anterior insular and orbitofrontal cortex (AIC and OFC, respectively) are known to play important roles in decision making under risk. However, risk-related AIC neural activity has not been investigated and it is controversial whether the rodent OFC conveys genuine risk signals. To address these issues, we examined AIC and OFC neuronal activity in rats responding to five distinct auditory cues predicting water reward with different probabilities. Both structures conveyed significant neural signals for reward, value and risk, with value and risk signals conjunctively coded. However, value signals were stronger and appeared earlier in the OFC, and many risk-coding OFC neurons responded only to the cue predicting certain (100%) reward. Also, AIC neurons tended to increase their activity for a prolonged time following a negative outcome and according to previously expected value. These results show that both the AIC and OFC convey neural signals related to reward uncertainty, but in different ways. The OFC might play an important role in encoding certain reward-biased, risk-modulated subjective value, whereas the AIC might convey prolonged negative outcome and disappointment signals.

Figure 1. Animal behavior and recording sites.

(a) Behavioral apparatus. N, nose poke hole; W, water-delivery nozzle. One of five different auditory cues, signaling five different reward probabilities (0, 25, 50, 75 and 100%), was delivered for 1 s during the animal’s nose poke (minimum 1 s to complete a trial). A blue LED (the small circle above the water-delivery nozzle) was turned on at nose poke exit, and the animal’s arrival at the reward port triggered probabilistic water delivery. (b) Behavioral performance. The proportion of complete trials (left), nose poke duration (middle), and reaction time (the time interval between nose poke exit and water-port entry; right) as a function of reward probability. The lines were determined with linear regression analyses. Error bars, SEM. (c) Single units were recorded from the AIC and lateral OFC. The photomicrographs are coronal sections of the rat brain showing marking lesions (arrows) in the AIC (left) and OFC (right). (d) The diagram is a coronal section view of the rat brain at 3.2 mm anterior to bregma. Each circle represents one recording site that was determined based on histology and electrode advancement history. Different colors represent recordings from different rats. One to five units were recorded simultaneously from each site. Modified with permission from Paxinos and Watson, 1998.

Figure 2. Neural signals for value, risk, and reward in the AIC and OFC.

(a) Two example neurons significantly responding to current value during the cue (left, OFC neuron) or response period (right, AIC neuron). Time 0 indicates the onset of the cue (‘Nose in’, left) or response period (‘Nose out’, right). Top, spike raster plots. Trial-by-trial neural activity is shown. Each row is one trial and each dot represents one spike. Trials were grouped according to value (i.e., reward probability). Middle, spike density functions. Neural activity was averaged across same-value trials after applying a Gaussian kernel (σ = 100 ms) to each spike. Darker colors indicate higher values. Bottom, mean (±SEM across trials) discharge rate during the cue (left) or response period (right) is plotted as a function of reward probability. (b) Two example neurons significantly responding to current risk during the cue (left, OFC neuron) or response period (right, AIC neuron). (c) Two example neurons significantly responding to current reward during the reward period (left, OFC neuron; right, AIC neuron). Time 0 indicates reward period onset. (d) Temporal profiles of neural signals for value, risk and reward. Shown are fractions of AIC and OFC neurons significantly responsive to value (reward probability), risk (standard deviation of the trial outcome) and reward (trial outcome) in the current (trial lag = 0) and two previous trials (trial lags = 1 and 2) in a 1-s analysis time window advanced in 0.1-s time steps across different behavioral periods (cue, response and reward periods). Vertical lines mark beginning (time 0) or end of a behavioral stage. Shading indicates mean chance level (binomial test, α = 0.05) for the AIC (7.7%) and OFC (7.0%). Large open circles indicate significantly different fractions (χ²-test, p < 0.05) between the AIC and OFC.

Figure 3. Characteristics of value-responsive neural activity.

(a) Same plots as in Fig. 2d except that value-responsive neurons were divided into activity-increasing (‘Positive’) and -decreasing (‘Negative’) types and plotted separately. Large open circles indicate significantly different fractions (χ²-test, p < 0.05) between activity-increasing and -decreasing types. (b) Population responses of value-responsive neurons. Mean discharge rates normalized to the peak firing rate of all value-responsive AIC and OFC neurons during the cue and response periods are plotted separately for activity-increasing (‘Positive’) and -decreasing (‘Negative’) types as a function of reward probability. The number in each plot indicates the number of neurons in each category and error bars denote SEM across neurons. Each line is an outcome of linear regression.

Figure 4. Characteristics of risk-responsive neural activity.

Same plots as in Fig. 3 except that risk-responsive neurons, instead of value-responsive neurons, were analyzed. (a) Risk-responsive neurons were divided into activity-increasing and-decreasing types and plotted separately. (b) Population responses of risk-responsive neurons during the cue and response periods plotted separately for activity-increasing and -decreasing types.

Figure 5. Neural responses concurrently coding value and risk.

(a) Four example neurons concurrently coding value and risk during the cue or response period. Same format as in Fig. 2a. (b) Population responses of all AIC and OFC neurons concurrently coding value and risk during the cue and response periods. The neurons were divided into four groups depending on their response directions (+ or −) to value (Q) and risk (σ). Same format as in Fig. 3b.

Figure 6. Characteristics of reward-responsive neural activity.

(a) Reward-responsive neurons were divided into activity-increasing and -decreasing types and plotted separately. Same format as in Fig. 3a. (b) Population responses of reward-responsive neurons. Mean peak-normalized discharge rates of all reward-responsive AIC and OFC neurons during the first 1 s of the reward period are plotted separately for activity-increasing and -decreasing types. Dark and light tones represent rewarded and unrewarded trials, respectively. Shading, 95% confidence interval. (c) Positive and negative reward signals in the AIC were examined for a prolonged time period aligning trials to the reward period onset. The dashed vertical line denotes the mean duration of unrewarded trials (1.704 s). (d) Positive and negative reward signals were compared between the AIC and OFC. Fractions of reward-responsive neurons are shown separately for activity-increasing (‘Positive’) and -decreasing (‘Negative’) types.

Figure 7. Activity of AIC neurons concurrently coding previous value and previous reward.

(a) An AIC example neuron concurrently coding value and reward in the previous trial during the 1-s time window centered around cue period onset. Top, a spike raster plot; Middle, spike density functions (σ = 100 ms); Bottom, mean (±SEM across trials) discharge rates during the 1-s time window. Trials were grouped according to value-reward interaction (value, 0, 1, 2, 3 and 4 for 0, 25, 50, 75 and 100% reward probability, respectively; reward, −1 and 1 for unrewarded and rewarded trials, respectively). (b) Mean (±SEM across neurons) peak normalized firing rates of all AIC neurons concurrently coding value and reward in the previous trial during the 1-s time window centered around cue period onset. The neurons were divided into four groups depending on their response directions (+ or −) to value (Q) and reward (R). No neuron belonged to the (−)Q(−)R group.