Action dominates valence in anticipatory representations in the human striatum and dopaminergic midbrain

Abstract

The acquisition of reward and the avoidance of punishment could logically be contingent on either emitting or withholding particular actions. However, the separate pathways in the striatum for go and no-go appear to violate this independence, instead coupling affect and effect. Respect for this interdependence has biased many studies of reward and punishment, so potential action-outcome valence interactions during anticipatory phases remain unexplored. In a functional magnetic resonance imaging study with healthy human volunteers, we manipulated subjects' requirement to emit or withhold an action independent from subsequent receipt of reward or avoidance of punishment. During anticipation, in the striatum and a lateral region within the substantia nigra/ventral tegmental area (SN/VTA), action representations dominated over valence representations. Moreover, we did not observe any representation associated with different state values through accumulation of outcomes, challenging a conventional and dominant association between these areas and state value representations. In contrast, a more medial sector of the SN/VTA responded preferentially to valence, with opposite signs depending on whether action was anticipated to be emitted or withheld. This dominant influence of action requires an enriched notion of opponency between reward and punishment.

Figure 1.

Experimental design. On each trial, one of four possible fractal images indicated the combination between action (making a button press in go trials or withholding a button press in no-go trials) and valence at outcome (win or lose). Actions were required in response to a circle that followed the fractal image after a variable delay. In go trials, subjects indicated via a button press on which side of the screen the circle appeared. In no-go trials, they withheld a response. After a brief delay, outcome was signaled in which a green upward arrow indicated a win of £1, a downward red arrow indicated a loss of £1, and a horizontal bar indicated the absence of a win or a loss. In go to win trials, a correct button press was rewarded. In go to avoid losing trials, a correct button press avoided punishment. In no-go to win trials, withholding a button press led to reward. In no-go to avoid losing trials, withholding a button press avoided punishment. The outcome was probabilistic so that 70% of correct responses were rewarded in win trials, and 70% of correct responses were not punished in lose trials. The red line indicates that half of the trials did not include the target detection task and the outcome. Subjects were trained in the task and fully learned the contingencies between the different fractal images and task requirements before scanning.

Figure 2.

Behavioral results. Mean percentage of trials in which subjects did a correct response within the response deadline for the go trials (blue) and did not emit any response on the no-go trials (red). Post hoc comparisons were implemented by means of repeated-measures t test: **p < 0.005.

Figure 3.

Response to anticipation of action and valence within anatomically defined ROIs in the striatum. Fractal images indicating go trials elicited higher activity than fractal images indicating no-go trials in all three bilateral ROIs: caudate, putamen, and ventral striatum (main effect of action, p < 0.05) (for details, see Table 1). In the right putamen, fractal images indicating go trials elicited higher activity than fractal images indicating no-go trials only in the lose conditions (action × valence interaction, p < 0.05) (for details, see Table 1).

Figure 4.

Voxel-based results within the striatum in response to anticipation of action and valence. A, Fractal images indicating go trials elicited higher dorsal striatal activity than fractal images indicating no-go trials (p < 0.001 uncorrected; p < 0.05 SVC within the whole-striatum ROI). The color scale indicates t values. B, Fractal images indicating go trials elicited higher ventral putamen activity than fractal images indicating no-go trials (p < 0.001 uncorrected; p < 0.05 SVC within the ROI restricted to the ventral striatum). The color scale indicates t values. C, Fractal images indicating that win trials elicited higher ventral putamen activity than fractal images indicating lose trials (p < 0.001 uncorrected; did not survive SVC within the ROI restricted to the ventral striatum). The color scale indicates t values. D, Parameter estimates at the peak coordinates confirm that activation of the ventral putamen signals the anticipation of action. Although the anticipation of valence also seems to have an effect in the left ventral putamen, the effect did not survive SVC. Coordinates are given in MNI space. Error bars indicate SEM (note that these parameter estimates were not used for statistical inference). L, Left; R, right.

Figure 5.

Midbrain response to anticipation of action and reward. A, Fractal images indicating that go trials elicited higher left lateral midbrain (or substantia nigra compacta) activity than fractal images indicating no-go trials (p < 0.001 uncorrected; p < 0.05 SVC). The color scale indicates t values. B, Parameter estimates at the peak coordinates in the left lateral midbrain confirm that activation at this location signals the anticipation of action regardless of the valence of the outcome of the action (reward or punishment avoidance). Coordinates are given in MNI space. Error bars indicate SEM (note that these parameter estimates were not used for statistical inference). C, An action × valence interaction was observed in the right medial midbrain or ventral tegmental area (p < 0.001 uncorrected; p < 0.05 SVC). The color scale indicates F values. D, Parameter estimates at the peak coordinates in the right medial midbrain confirm that activation at this location signals the anticipation of action if the outcome of the action is rewarding. The anticipation of actions that avoid punishment, conversely, is associated with a relative deactivation of this region. The inverse pattern of activation is observed for the no-go trials: the anticipation of a passive response that wins a reward is associated with relative deactivation, whereas the anticipation of a passive response that avoids punishment is associated with activation. Coordinates are given in MNI space. Error bars indicate SEM (note that these parameter estimates were not used for statistical inference). L, Left; R, right.

Figure 6.

Brain responses to the outcome. A, Activation in the ventral striatum revealed by a one-way ANOVA with valence as factor (p < 0.001 uncorrected; p < 0.05 SVC). The color scale indicates F values. B, Post hoc t test on the peak voxel in ventral striatum revealed that the main effect was driven by higher activation for the win trials when compared with the loss and neutral trials (*p < 0.005; **p < 0.001). C, Activation in left medial prefrontal cortex revealed by a one-way ANOVA with valence as factor (p < 0.001 uncorrected; p < 0.05 whole-brain FWE). The color scale indicates F values. D, Post hoc t test on the peak voxel in left medial prefrontal cortex revealed a main effect driven by greater activation for win compared with loss and neutral trials, respectively (**p < 0.001). E, Activation in right insula revealed by a one-way ANOVA with valence as factor (p < 0.001 uncorrected; p < 0.05 whole-brain FWE). The color scale indicates F values. F, Post hoc t test on the peak voxel in the right insula revealed that the main effect was driven by higher activation for loss trials compared with win and neutral trials (*p < 0.005; **p < 0.001). L, Left; R, right.