Retroactive and graded prioritization of memory by reward

Abstract

Many decisions are based on an internal model of the world. Yet, how such a model is constructed from experience and represented in memory remains unknown. We test the hypothesis that reward shapes memory for sequences of events by retroactively prioritizing memory for objects as a function of their distance from reward. Human participants encountered neutral objects while exploring a series of mazes for reward. Across six data sets, we find that reward systematically modulates memory for neutral objects, retroactively prioritizing memory for objects closest to the reward. This effect of reward on memory emerges only after a 24-hour delay and is stronger for mazes followed by a longer rest interval, suggesting a role for post-reward replay and overnight consolidation, as predicted by neurobiological data in animals. These findings demonstrate that reward retroactively prioritizes memory along a sequential gradient, consistent with the role of memory in supporting adaptive decision-making.

Fig. 1

Experimental design to test the retroactive effect of reward on memory. The task consisted of maze exploration followed by a surprise memory test. a In the maze exploration phase, participants explored a series of mazes searching for a gold coin (each worth a bonus $1; 2007 Presidential $1 Coin image from the United States Mint). Mazes varied in length, ranging from 3 to 15 steps. Participants explored a total of 22 mazes. Half of the mazes ended in reward and half ended with no reward. During exploration, participants encountered trial-unique object pictures, appearing one at a time in the participants’ current location. The objects were not related to the maze outcome and each object only appeared in one location within one maze during the entire task. The outcome of the maze was not known at the time the object was presented; therefore, any effect of reward on object memory must be due to retroactive modulation. A fixation interval of 15, 20 or 25 s followed each maze. During this time, participants were instructed to rest. b At each step, participants had 2 s to choose which adjacent square to navigate to next, and after taking that step, they saw a picture of an object appear in the chosen location for 2.5 s, after which the object was replaced with a white square for a brief interval. After that, the square turned grey, indicating that the participant should make his or her next navigational choice. c In the second phase, participants were given a surprise memory test after a delay of either 15 minutes or 24 hours. Participants were presented with a series of objects, one at a time, and were asked to indicate if the object was old (presented during the maze) or new (a lure) and the confidence of their response

Fig. 2

Selective retroactive modulation of memory by reward and proximity (Experiment 1). a 24-hour condition (n = 23). Rewards retroactively modulated memory, such that participants were more likely to remember objects that were more proximal to the reward. The top panel depicts the model predictions, showing how the proximity of the object was positively related to participants’ memory for objects in rewarded vs. unrewarded mazes. The middle panel depicts the beta coefficients for the reward and no-reward conditions. The error bars represent the standard error of the reward × proximity interaction, and the dot plot overlay shows the reward and no-reward betas for each participant. The bottom panel depicts the interaction term representing the difference between the slopes in the reward and no-reward condition by proximity; the group level is shown in black, and individual participants are in light grey. b 15-minute condition (n = 21). For this condition there was no evidence for a reward by proximity interaction and a significant interaction with delay condition was observed, indicating a significantly greater reward proximity memory effect in the 24-hour condition. The top panel depicts the model predictions, showing the relationship between the proximity of the object and participants’ memory for objects in rewarded vs. unrewarded mazes. The middle panel depicts the beta coefficients for the reward and no-reward conditions. The error bars represent the standard error of the reward × proximity interaction, and the dot plot overlay shows the reward and no-reward betas for each participants. The bottom panel depicts the interaction term representing the difference between the slopes in the reward and no-reward condition by proximity; the group level is shown with the black line and individual participants are in light grey

Fig. 3

Reward proximity effect increases with longer post-encoding rest. (Experiment 1). a In the 24-hour condition (n = 23), we found that the duration of the rest break following each maze modulated the reward proximity effect, such that the interaction was stronger if the rest break following the maze was longer. b In the 15-minute condition (n = 21), we did not find an effect of the duration of the rest break modulating the reward by proximity interaction. The direct comparison of the 24-hour and 15-minute conditions showed a significant interaction (see Supplementary Table 1b for post hoc tests). The insets depict the beta coefficients for the reward and no-reward conditions. The error bars represent the standard error of the reward × proximity interaction, and the dot plot overlay shows the reward and no-reward betas across participants

Fig. 4

Reward proximity effect is not due to strategic rehearsal. (Experiment 3) To rule out the possibility that the reward proximity effect was due to strategic rehearsal during the rest intervals, we conducted a follow-up experiment in which participants completed one of three distractor tasks during the rest breaks. a In the target detection condition (n = 27), participants were instructed to make a key response every time a target image appeared in a maze square, but not when a lure image (a dark grey square) appeared (top). Despite the distractor task, we found that rewards retroactively modulated memory, such that participants were more likely to remember objects that were more proximal to the reward (bottom). b In the navigation condition (n = 27), participants used the arrow keys to navigate to a target (top). We again replicated the reward proximity effect. c In the working memory condition (n = 23), participants were presented with a target (a configuration of four randomly chosen colours in four randomly chosen squares) at the beginning of the rest interval and were instructed to make a key response to every presentation of this target configuration, but no responses to non-target configurations (other combinations of colours and squares) (top). In the working memory condition, we again replicated the reward proximity effect (bottom). We did not find a significant effect of distractor condition on the reward by proximity interaction (target detection condition vs. navigation condition × reward × proximity; target detection condition vs. working memory condition × reward × proximity (see Supplementary Table 3a for post hoc tests). The insets depict the beta coefficients for the reward and no-reward conditions. The error bars represent the standard error of the reward × proximity interaction and the dot plot overlay shows the reward and no-reward betas across participants

Fig. 5

Rewards retroactively modulate spatial location memory. a Schematic of surprise spatial location memory test. An old object was randomly placed in a maze square, and participants were instructed to move the object back to the square where they originally saw the object by using arrow keys to move through the maze and then pressing the space bar to indicate the chosen location (self-paced). b Reward retroactively modulated spatial memory for sequentially proximal objects such that spatial location memory decreased as proximity to the end of the maze increased for the no (or low) reward mazes (Experiments 1–4 combined, 24-hour conditions only, n = 146). The inset depicts the beta coefficients for the reward and no-reward conditions. The error bars represent the standard error of the reward × proximity interaction and the dot plot overlay shows the reward and no-reward betas across participants