Beyond working memory: the role of persistent activity in decision making

Abstract

Since its first discovery in the prefrontal cortex, persistent activity during the interval between a transient sensory stimulus and a subsequent behavioral response has been identified in many cortical and subcortical areas. Such persistent activity is thought to reflect the maintenance of working memory representations that bridge past events with future contingent plans. Indeed, the term persistent activity is sometimes used interchangeably with working memory. In this review, we argue that persistent activity observed broadly across many cortical and subcortical areas reflects not only working memory maintenance, but also a variety of other cognitive processes, including perceptual and reward-based decision making.

Figure 1

Persistent activity for spatial working memory. (a) Left, lateral view of the rhesus monkey brain. Right, spike density functions of a single neuron in the dorsolateral prefrontal cortex of a rhesus monkey during the delay period (gray background), with colors indicating the remembered location of the visual stimulus (see inset) briefly flashed during the cue period (yellow background). During the experiment, all peripheral stimuli were presented in the same color [20,48]. (b) Left, maintenance of a spatial location during working memory task-evoked BOLD activity in the frontal and parietal cortices, shown as a statistical map overlaid on the inflated cortex (sulci, dark gray; gyri, white). Top right, a BOLD signal persisted during a spatial working memory task in the dorsolateral prefrontal cortex (dotted circle in the left panel) and was greater for memoranda in the contralateral visual field (solid line) than for those in the ipsilateral field (dashed line). The yellow bar represents presentation of the sample cue and the gray background depicts the memory delay. Bottom right, time course of BOLD signals from the dorsolateral prefrontal cortex aligned on presentation of the sample cue during a spatial working memory task. Separate lines represent the different delay lengths (indicated by colored bars at the bottom). Importantly, persistent activity bracketed by the phasic BOLD signals increased in duration with the delay length and was sustained until the working memory representation could be used to guide the response [10].

Figure 2

Persistent activity resulting from temporal integration can subserve multiple types of cognitive processes, including working memory, accumulation of noisy sensory evidence and creation of eligibility trace and other computations in reinforcement learning. (a,b) Schematic illustration of (a) the time course of inputs leading to persistent activity during a working memory task and (b) buildup activity related to the accumulation of noisy sensory inputs. (c) Hypothetical signals related to the animal's actions (green and blue vertical bars for leftward and rightward actions, respectively) can be temporally integrated to generate eligibility traces (upper and middle left panels), whereas temporal integration of reward (red disks) can lead to signals related to average reward rate (bottom left panel). The value function for each action is incremented by the reward prediction error, namely, the discrepancy between the reward earned and the reward predicted from the current value function, weighted by its eligibility trace. As a result, reward delivered in a given trial increases the value functions according to the recency of each action (upper and middle right panels), even when it was not chosen in the last trial. Dark red disks and colored vertical bars indicate the trials in which the leftward or rightward action led to immediate reward. The difference in value functions for the two alternative actions is also shown [bottom right panel; dark red circles indicate the trials in which reward was obtained after choosing the leftward (top) and rightward (bottom) actions, respectively].

Figure 3

Persistent activity for decision making and reinforcement learning. (a) Persistent activity related to the animal's choice [20]. The spike density function of the same neuron illustrated in Figure 1a during a binary decision-making task is plotted separately according to the position of the target chosen in the current trial (trial lag=0) or in each of the previous three trials (trial lag=1–3). Black and green lines indicate the activity for the trials in which the animal chose the leftward and rightward targets, respectively. (b) Persistent activity related to the outcome of the animal's decision [43]. The spike density functions of a neuron in the anterior cingulate cortex is plotted separately according to whether the animal was rewarded (green) or not (black) in the current (trial lag=0) or in each of the previous three trials (trial lag=1–3) using the same format as in (a). (c) Proportion of neurons in the dorsolateral prefrontal cortex (DLPFC), anterior cingulate cortex (ACC) and posterior parietal cortex (lateral intra-parietal area, LIP) that showed a significant change in activity according to the animal's choice (top), the choice of the computer opponent (middle), and outcome of the animal's choice (bottom) in the current (trial lag=0) and previous three trials (trial lag=1–3). Two subpanels for each trial lag illustrate the activity aligned at the target onset (left) and feedback onset (right), and gray bars correspond to the 0.5-s cue period and feedback period, respectively. The decision-making task used in these studies simulated a matching-pennies task in which the animal was rewarded only when it chose the same target as the computer opponent [,,–49,53].