Action-specific value signals in reward-related regions of the human brain

Abstract

Estimating the value of potential actions is crucial for learning and adaptive behavior. We know little about how the human brain represents action-specific value outside of motor areas. This is, in part, due to a difficulty in detecting the neural correlates of value using conventional (region of interest) functional magnetic resonance imaging (fMRI) analyses, due to a potential distributed representation of value. We address this limitation by applying a recently developed multivariate decoding method to high-resolution fMRI data in subjects performing an instrumental learning task. We found evidence for action-specific value signals in circumscribed regions, specifically ventromedial prefrontal cortex, putamen, thalamus, and insula cortex. In contrast, action-independent value signals were more widely represented across a large set of brain areas. Using multivariate Bayesian model comparison, we formally tested whether value-specific responses are spatially distributed or coherent. We found strong evidence that both action-specific and action-independent value signals are represented in a distributed fashion. Our results suggest that a surprisingly large number of classical reward-related areas contain distributed representations of action-specific values, representations that are likely to mediate between reward and adaptive behavior.

Figure 1.

A, Instrumental learning task. Subjects were presented with an arbitrary stimulus and had 2500 ms to make one of two responses via a button box in either hand. Each stimulus–action pairing was associated with a certain probability of reward (10 pence) versus no reward. Outcomes were signaled with two different sounds, which were presented for 1000 ms, followed by a variable intertrial interval (1000–3000 ms, uniform distribution). B, Proportion of trials on which subjects chose the objectively higher-valued action, pooled over all subjects, sessions, and cues. Subjects chose the objectively better action on an increasing proportion of trials—showing that they were able to learn the task contingencies. Error bars indicate bootstrapped 95% confidence intervals. C, Schematic of distributed and coherent coding schemes on a two-dimensional surface. Under a distributed encoding scheme (left), nearby voxels do not show similar responses, in contrast to a scheme with local coherence (right), where clumping of responses is observed. Red/orange, positive response to arbitrary parameter; blue, negative response; green, no response.

Figure 2.

Voxel weights (η) from the MVB analysis for AV in the right vmPFC ROI for Subject 5. A, Voxels with positive (red) and negative (blue) voxel weights overlaid on Subject 5's T1-weighted structural scan (x = 3 mm). Voxels with a positive weight show a positive response to AV and voxels with a negative weight show a negative response to AV according to our multivariate analysis. Image thresholded at η > 0.00005 for positive weights and η < 0.00005 for negative weights. Positive and negative weights are interspersed without any obvious pattern, suggesting a lack of spatial coherence. B, Histogram of voxel weights. Only a small proportion of voxels have large weights, showing that only a small proportion are important for decoding (this is the hallmark of sparse distribution).