Anterior cingulate engagement in a foraging context reflects choice difficulty, not foraging value

Abstract

Previous theories predict that human dorsal anterior cingulate (dACC) should respond to decision difficulty. An alternative theory has been recently advanced that proposes that dACC evolved to represent the value of 'non-default', foraging behavior, calling into question its role in choice difficulty. However, this new theory does not take into account that choosing whether or not to pursue foraging-like behavior can also be more difficult than simply resorting to a default. The results of two neuroimaging experiments show that dACC is only associated with foraging value when foraging value is confounded with choice difficulty; when the two are dissociated, dACC engagement is only explained by choice difficulty, and not the value of foraging. In addition to refuting this new theory, our studies help to formalize a fundamental connection between choice difficulty and foraging-like decisions, while also prescribing a solution for a common pitfall in studies of reward-based decision making.

Figure 1. The role of choice difficulty in a standard foraging setting

In a typical patch-leaving scenario, an animal faces the recurring decision whether to continue harvesting a patch with decreasing marginal returns (e.g. fewer ripe fruits; blue line) or leave for a new patch. The expected value of switching to a new patch (green line) accounts for the expected reward in the new patch, as well as the travel time between patches (gray region). An optimally foraging animal will exit the patch at the point where blue and green lines meet (dashed horizontal black line), which is also their indifference point between these options. Dashed blue and green lines indicate theoretical values of staying and switching that an optimally foraging animal typically would not encounter (having already departed the patch), but that could theoretically be examined with a task like KBMR’s that examines cross-sections through a foraging-like context (see Fig. 2). For simplicity, here we assume a situation where the value of each new patch, the (exponential) reward decay rate, and the travel time to a new patch remain constant across patches.

Figure 2. KBMR’s estimates of value and choice difficulty do not align with behavioral data

a) Schematic of an example trial in KBMR’s task. In Stage 1 (upper panels), participants are offered a pair of potential rewards (large numbers). They can choose to forage for a better pair of rewards from the set shown at the top of the screen (smaller numbers in the red box), in which case a random pair from that set is swapped with the current offer and they incur a forage cost (shown at left below red box) and a delay until the new choice is shown. They can forage as many times as they prefer (or not at all) before opting to proceed to Stage 2 (lower panel) and engage in the selected choice. At that point, a probability is randomly assigned to each reward (height of violet bar beside each number), and they choose which reward-probability pair to attempt. They receive the outcome of this gamble as points that accumulate at the bottom of the screen (not shown). While potential rewards were indicated numerically in Experiment 2 (as shown here), abstract symbols with learned reward associations were used in the original task and Experiment 1. b–c) Choice (left panels) and RT data (right panels) from the two stages of Experiment 1 (black curves). Gray bars show the histograms of trial frequencies. In Stage 1 (b), both the indifference point in the choice curve and the peak in response times exhibit a clear (and comparable) shift to the right of RV_forage = 0. In contrast, in Stage 2 (c), both the indifference point and RT peak coincide with RV_right = 0. Red curves in each figure show the predicted RTs and choice probabilities, and corresponding indifference points (vertical dotted lines) based on fits of the decision making model (see Methods). We corrected RV_forage so that it was centered on this empirical indifference point (RV_forage-C = 0) and defined choice difficulty as value similarity with respect to this corrected measure (−|RV_forage-C|; green-red shading). These data further show that in Experiment 1, as in KBMR’s study, the indifference point occurs toward the higher end of forage values tested, confounding forage value and choice difficulty. For display purposes panels b–c (and Fig. 4b) show the result of a fixed-effects model across all participants (error bars reflect s.e.m), but all analyses reported in the main text were based on individual participant fits. Note also that continuous data were used in all fits but are shown here binned. We have also truncated the x-axis to only show RV_forage bins with an average of five or more trials per participant, but show complete fits in Supplementary Fig. 2.

Figure 3. Experiment 1: Choice difficulty accounts for dACC activation in both stages of KBMR’s original task

a) Whole-brain contrast for brain regions tracking KBMR’s estimate of the relative value of forage vs. engage options in Stage 1 (RV_forage) (top), and the chosen vs. unchosen option in Stage 2 (bottom). b) Regions tracking the similarity of option values (i.e. −|RV_forage|) for the same options represented in Panel a. We replicate the finding of significant dACC activity in the contrasts shown in the top panel of a and bottom panel of b (indicated with green arrow), consistent with the foraging value account. c) However, using an estimate of value similarity corrected to align with the behavioral data (i.e., −|RV_forage-C| in Stage 1), the same region of dACC is found to be associated with choice difficulty in both Stage 1 and Stage 2 (red arrows). A conjunction of these two contrasts (shown in the center) indicates a large degree of overlap in dACC. Statistical maps in a–c are thresholded at voxelwise p<0.01, extent threshold of 200 voxels.

Figure 4. Experiment 2 de-confounds foraging value and choice difficulty

a) Because of the original confound (Fig. 2b), forage value (green) and choice difficulty (red) accounts make similar predictions with respect to dACC activity in Experiment 1 (Fig. 3). However, Experiment 2 dissociates the two accounts: foraging value predicts that dACC should increase monotonically, whereas choice difficulty predicts that dACC should decrease as foraging value increases beyond the indifference point (Fig. 5). b) Like Experiment 1, behavior and model fits from Experiment 2 exhibited a shift in the indifference point in Stage 1 (compare Fig. 2b), but a wider range of choices allowed us to de-confound difficulty and RV_forage.

Figure 5. Experiment 2: Choice difficulty but not foraging value accounts for dACC activation when a wider range of foraging values is used

a) Average BOLD activity for each of six RV_forage quantiles in Experiment 1, taken from a dACC region-of-interest (ROI) around peak coordinates from the contrast shown in Fig. 3a, top panel. This is provided for visual reference but note that, unlike the remaining panels, this analysis is circular because it is intentionally biased toward the dACC region that is maximally sensitive to RV_forage. b) Given the high correlation between foraging value and choice difficulty in Experiment 1, the same pattern of activity is observed when dACC activity is binned by choice difficulty (error bars, between-subject s.e.m). c–d) Results from Experiment 2, using a wider range of foraging values that orthogonalized this with respect to choice difficulty. Images in panel c show whole-brain contrasts during Stage 1, showing that dACC activity exhibits a quadratic but not linear relationship to RV_forage; plot in bottom panel confirms that, over the fuller range of RV_forage values used, dACC exhibits the non-monotonic pattern of activity predicted by the choice difficulty account (compare Fig. 4a; also see Supplementary Figs. 3 and 8). For ease of comparison with contrasts in d, the color map for the quadratic contrast is inverted so that negative coefficients (suggesting an inverted U-shape) appear in red-yellow. Panel d shows that a whole brain contrast for a linear relationship with choice difficulty again identifies dACC, plotted in bottom panel; inset shows conjunction with the same contrast for Stage 2. *This contrast is shown at a liberal voxelwise threshold of p<0.05, no cluster extent threshold. All other statistical maps are shown at voxelwise p<0.01, extent threshold of 200 voxels.