What does dopamine mean?

Abstract

Dopamine is a critical modulator of both learning and motivation. This presents a problem: how can target cells know whether increased dopamine is a signal to learn or to move? It is often presumed that motivation involves slow ('tonic') dopamine changes, while fast ('phasic') dopamine fluctuations convey reward prediction errors for learning. Yet recent studies have shown that dopamine conveys motivational value and promotes movement even on subsecond timescales. Here I describe an alternative account of how dopamine regulates ongoing behavior. Dopamine release related to motivation is rapidly and locally sculpted by receptors on dopamine terminals, independently from dopamine cell firing. Target neurons abruptly switch between learning and performance modes, with striatal cholinergic interneurons providing one candidate switch mechanism. The behavioral impact of dopamine varies by subregion, but in each case dopamine provides a dynamic estimate of whether it is worth expending a limited internal resource, such as energy, attention, or time.

FIG. 1.. Dopamine: updating the past, invigorating the present.

Top, Circles with arrows represent states and the potential actions from those states. Arrow widths indicate learned values of performing each action. As states/actions fade into the past, they are progressively less eligible for reinforcement. Middle, a burst of dopamine occurs. The result is invigoration of actions available from the current state (red), and plasticity of the value representations for recently performed actions (purple). Bottom, as the result of plasticity, the next time these states are encountered their associated values have increased (arrow widths). Through repeated experience reinforcement learning can “carve a groove” through state space, making certain trajectories increasingly more likely. In addition to this learning role, the invigorating, performance role of dopamine seems to speed up the flow along previously-learned trajectories.

FIG. 2.. Fast dopamine fluctuations signal dynamically-evolving reward expectations.

a-c) Mesolimbic dopamine release rapidly increases as rats get closer to anticipated rewards. d) Value, defined as temporally-discounted estimates of future reward, increases as reward gets closer. Cues indicating that reward is larger, closer, or more certain than previously expected cause jumps in value. These jumps from one moment to the next are temporal-difference RPEs. e) Subtracting away “baselines” can confound value and RPE signals. Left, dopamine aligned to reward-predictive cue(at time zero), with conventional baseline subtraction, seems to show that dopamine jumps to higher levels when reward is less expected(brown), resembling an RPE signal. Right, an alternative presentation of the same data, equating dopamine levels after the cue, would show instead that precue dopamine levels depend on reward expectation(value). Additional analyses determined that the right-side presentation is closer to the truth(see details in ref. 15). Panel a reproduced, with permission, from ref , Macmillan Publishers Limited….; panel b reproduced, with permission, from ref. , Elsevier; panels c-e reproduced, with permission, from ref , Macmillan Publishers Limited