Behavioral and neurophysiological correlates of regret in rat decision-making on a neuroeconomic task

Abstract

Disappointment entails the recognition that one did not get the value expected. In contrast, regret entails recognition that an alternative (counterfactual) action would have produced a more valued outcome. In humans, the orbitofrontal cortex is active during expressions of regret, and humans with damage to the orbitofrontal cortex do not express regret. In rats and nonhuman primates, both the orbitofrontal cortex and the ventral striatum have been implicated in reward computations. We recorded neural ensembles from orbitofrontal cortex and ventral striatum in rats encountering wait or skip choices for delayed delivery of different flavors using an economic framework. Economically, encountering a high-cost choice after skipping a low-cost choice should induce regret. In these situations, rats looked backwards toward the lost option, cells within orbitofrontal cortex and ventral striatum represented the missed action, rats were more likely to wait for the long delay, and rats rushed through eating the food after that delay.

Figure 1. Restaurant Row and revealed preferences in rats

a, The Restaurant Row task consisted of a central ring with four connected spokes leading to individual food flavors. Rats ran around the ring, encountering the four invisible zones (square boxes) sequentially. Color reflects flavor: magenta=cherry, yellow=banana, black= nonflavored/plain, brown=chocolate. b–e, Rats typically waited through short delays but skipped long delays. Each panel shows the stay/go decisions for all encounters of a single rat running a single session (R210-2011-02-02). A small vertical jitter has been added for display purposes. Thresholds were fit as described in Methods. f–i, Each rat demonstrated a different revealed preference that was consistent within rat across all sessions, but differed between rats. Thresholds were fit for each flavor for each session. Each panel shows the mean fit threshold for a given rat, with standard error shown over sessions. An important consideration is to control for the possibility that rats were waiting for a specific cue before leaving the zone. The fact that rats either stayed through the entire delay or left after a very stable 3 seconds implies that rats were not waiting for a specific delay cue, but were making economic decisions based on the delay offered. (See Supplemental Fig S1).

Figure 2. Ensembles in OFC and vStr represent the current reward and the current zone

a–b, p(Reward) at Reward, Defining the training set for decoding as activity at reward delivery and the test set as activity at each moment surrounding reward-delivery (shaded area represents standard error), the neural ensemble decoded the current reward reliably (distribution of current reward was determined to be significantly different, empirical cumulative distribution function, alpha = 0.05). p(Reward) is the posterior probability indicating representation of a given reward flavor as calculated by Bayesian decoding. c, Cartoon indicating that the training set is the reward types, and the test set is activity when the rat receives reward. d–e, p(Zone) at Zone, Defining the training set for decoding as neuronal activity at zone entry and the test set as neuronal activity at each moment surrounding zone-entry, the neural ensemble decoded the current zone reliably. p(Zone) is the posterior probability indicating representation of a given zone entry as calculated by Bayesian decoding. f, Cartoon indicating that the training set is zone entry, and the test set is neuronal activity when the rat enters the zone, triggering the cue that signals the delay. g–h, p(Reward) at Zone, Defining the training set for decoding as neuronal activity at reward-delivery and the test set as neuronal activity at each moment surrounding zone entry, the neural ensemble at time of zone entry decoded the current reward type reliably. i, Cartoon indicating that the training set is the reward flavor, and the test set is neuronal activity when the rat enters the zone, triggering the cue/tone.

Figure 3. Representations of expected reward as a function of delay and threshold

In order to determine whether orbitofrontal (OFC) and ventral striatal (vStr) signals predicted behavior at time of zone-entry, we measured p(Reward) at Zone for all offers above and below the threshold for a given rat for a given flavor-reward-site (shaded area represents standard error). a,b, Low-cost offers in which the rat waited through the delay (distribution of current reward was determined to be significantly different, empirical cumulative distribution function, alpha = 0.05). c,d, High-cost offers in which the rat skipped out and did not wait through the full delay. a,c, OFC. b,d, vStr. e, Cartoon indicating that this decoding operation was based on a training set at the reward, but a test-set at zone-entry.

Figure 4. Behavioral responses in regret-inducing and control situations

All passes were rotated so as to align on entry into a “current” zone. Orientation was measured using the curvature measure as per Methods. a–c, examples of approaches for each of the three conditions: regret-inducing, control 1 (same sequence, took previous option), and control 2 (two long delays in a row). a, In a regret-inducing example, when the animal entered the zone, he paused and looked backwards towards the previous zone. b, In a control 1 example, the animal looked towards the current reward spoke, but proceeded on to the next zone. c, In a control 2 example, the animal looked towards the next zone, but turned back towards the current reward. d–f, Summary statistics. The first re-orientation event was measured as per Methods. Grey traces show all pausing re-orientations over all instances within that condition. Heavy line shows vector average within each 120 degree arc. d, In the regret-inducing conditions, rats tended to orient towards the previous zone or current spoke. e, In the control 1 conditions, rats tended to orient only towards the current spoke. f, In the control 2 conditions, rats tended to orient towards the next zone. The distributions in d, e, and f, were significantly different from each other (Watson’s Circular U, see text).

Figure 5. Single reward cells in OFC and vStr during regret-inducing situations

Top Panel. OFC Example Cell during regret-inducing situation. Grey dots represent individual spikes. Solid colored lines indicate Gaussian smoothed activity, sigma = 50ms. Black = nonflavored pellets, magenta = cherry flavored, yellow = banana flavored, brown = chocolate flavored. Black dots in the center panel represent behavioral samples during this particular instance. Red dots show spikes aligned to behavior. The rat traveled in a counterclockwise direction. The maze has been aligned so that the current zone is represented by the bottom right zone. This particular cell responded most to entry into the cherry reward zone, and little to the banana reward zone. When the rat skipped a low cost cherry zone opportunity and encountered a high cost banana zone opportunity, the rat looked back towards the previous reward; and the activity of the cell approximated that of the cherry-zone-entry response. Bottom Panel. Display same as top panel, vStr example cell during a regret-inducing situation from the chocolate-reward zone to the cherry-reward zone.

Figure 6. Neural representations in OFC and vStr represent the previous zone during behavioral regret instances

In regret-inducing conditions, the p(Zone) representation of the previous encounter was high after zone entry into the current zone for both OFC (a) and vStr (b) (shaded areas represent standard error). Green traces show decoding using shuffled ISIs. Decoding to the previous zone was significantly different from all other conditions, even after controlling for multiple comparisons (ANOVA, OFC: p << 0.001; vStr: p << 0.001, distribution significantly different as determined by empirical cumulative distribution function, alpha = 0.05). Panel c shows a cartoon of the conditions being decoded – the rat has skipped the previous offer, even though the delay was less than threshold for that restaurant, and has now encountered a delay greater than threshold for the current restaurant. In the control 1 condition, p(Zone) representation of the current zone increased until the rat heard the cue indicating a long delay, at which time, the representation changed to reflect the next zone. In control 1, p(Zone) representations to the current and next zones were significantly different from the other zones (ANOVA, vStr: p << 0.001; OFC: p << 0.001), although they were not different from each other after controlling for multiple comparisons (ANOVA, vStr: p = 0.074, OFC: p = 0.619). (d: OFC; e: vStr; f: Cartoon indicating condition.) In the control 2 condition, p(Zone) representation of both the current and previous zones was increased when the rat heard the cue indicating a long delay (compared to other zones ANOVA, OFC: p << 0.001, vStr p << 0.001). (g: OFC; h: vStr; i: Cartoon indicating condition.) Decoding to the current and previous zones in control 2 were not significantly different from each other (ANOVA, OFC: p = 0.509; vStr: p = 0.268).

Figure 7. Behavioral changes following potential regret instances

a, Comparing the proportion of stays to skips during each condition revealed that rats were significantly more willing to wait for reward following regret-inducing instances compared to control 1 instances (Wilcoxon, p = 0.01) or control 2 instances (Wilcoxon, p = 0.06). b, Rats spent less time consuming reward during regret than during non-regret instances. Typical handling time mean = 25.3 s, standard deviation = 12.2 s, regret handling time mean = 15.2 s, standard deviation 14.2 s, control handling times are distributed the same as all non-regret handling times.

Figure 8. Behavioral and neurophysiological correspondences during regret

In order to determine whether the representations of previous reward were different when the rat chose to stay at the high-delay (high-cost) current zone, we measured the ratio between the p(Zone) representation of the previous zone against the p(Zone) representation of the current zone from 0 to 3 seconds following zone entry for all conditions in the event that the rat skipped or stayed. Each panel shows a box plot of the distribution of p(Zone_previous)/p(Zone_current) ratios divided between stays and skips. Box limits are 25th and 75th percentiles, whiskers extend to data not considered outliers and outliers are plotted separately. a, p(Zone_previous)/p(Zone_current) ratios from OFC ensembles during regret-inducing conditions. b, p(Zone_previous)/p(Zone_current) ratios from vStr ensembles during regret-inducing conditions. d, e, during control 1 conditions. f, g, during control 2 conditions. Following regret inducing instances, when rats were more willing to wait for reward, p(Zone_previous) was greater than p(Zone_current).