Context, emotion, and the strategic pursuit of goals: interactions among multiple brain systems controlling motivated behavior

Abstract

Motivated behavior exhibits properties that change with experience and partially dissociate among a number of brain structures. Here, we review evidence from rodent experiments demonstrating that multiple brain systems acquire information in parallel and either cooperate or compete for behavioral control. We propose a conceptual model of systems interaction wherein a ventral emotional memory network involving ventral striatum (VS), amygdala, ventral hippocampus, and ventromedial prefrontal cortex triages behavioral responding to stimuli according to their associated affective outcomes. This system engages autonomic and postural responding (avoiding, ignoring, approaching) in accordance with associated stimulus valence (negative, neutral, positive), but does not engage particular operant responses. Rather, this emotional system suppresses or invigorates actions that are selected through competition between goal-directed control involving dorsomedial striatum (DMS) and habitual control involving dorsolateral striatum (DLS). The hippocampus provides contextual specificity to the emotional system, and provides an information rich input to the goal-directed system for navigation and discriminations involving ambiguous contexts, complex sensory configurations, or temporal ordering. The rapid acquisition and high capacity for episodic associations in the emotional system may unburden the more complex goal-directed system and reduce interference in the habit system from processing contingencies of neutral stimuli. Interactions among these systems likely involve inhibitory mechanisms and neuromodulation in the striatum to form a dominant response strategy. Innate traits, training methods, and task demands contribute to the nature of these interactions, which can include incidental learning in non-dominant systems. Addition of these features to reinforcement learning models of decision-making may better align theoretical predictions with behavioral and neural correlates in animals.

Keywords: Pavlovian-instrumental transfer; amygdala; dopamine; emotion; hippocampus; inhibition; reinforcement learning; striatum.

Figure 1

Diagram illustrating select connectivity of brain structures involved in the voluntary control of rodent behavior. Four regions of rodent striatum are indicated by labels and color; the color gradient approximates the gradient of afferent projections (Voorn et al., 2004). Corresponding color in the other structures represents general projection topography. Note that ventral hippocampus has direct projections outside of the hippocampal formation, whereas the dorsal hippocampus does not (see text). Tapered arrows indicate highly convergent input. Output of the striatum can proceed through two distinct pathways to reach the thalamus and other targets: a “direct” pathway that has a disinhibitory effect (+), and an indirect pathway that has an inhibitory effect (−). These pathways are innervated by separate populations of medium spiny neurons in the striatum that predominantly express D1 (direct) or D2 (indirect) dopamine receptors. Projections from dopamine neurons in the VTA and SNc are shown in red. The overarching organization is that of loops linking neocortex, basal ganglia, and thalamus. The table indicates some characteristic features of dissociated behavioral control systems, with color indicating corresponding brain structures. Abbreviations for the table: stimulus (S), context (C), affective outcome (Oa), response (R), specific outcome (O). Abbreviations for the main figure: DLS, dorsolateral striatum; DMS, dorsomedial striatum; VSc, core of the nucleus accumbens in ventral striatum; VSs, shell of the nucleus accumbens in ventral striatum; VTA, ventral tegmental area; SNc, substantia nigra pars compacta, SNr, substantia nigra pars reticulata, P, pallidum; STN, subthalamic nucleus; dH, dorsal hippocampus; vH, ventral hippocampus; ENT, entorhinal cortex; BLA, basolateral nucleus of the amygdala; CN, central nucleus of the amygdala. The following regions of neocortex are labeled: IL, infralimbic; PL, prelimbic; OF, orbitofrontal; CG, cingulate, PP, parietal; SMA, sensorimotor.

Figure 2

Training and performance on the cue/place water task. (A) Rats are trained for three days to swim from one of four start points to a visible platform located in the same spatial position relative to the pool and room cues. On the fourth day, the visible platform is removed and an invisible (submerged) platform is put in its place. This sequence is repeated three times so that each animal receives nine visible platform days of training and three invisible training days. After training, a competition test is performed in which the visible platform is moved to a new location in the pool and the invisible platform remains in the original position. Rats start from one of two points equidistant from the platforms. (B) On this competition test, control rats are equally likely to choose either the cue or the place response. Rats with neurotoxic damage to the hippocampus mainly swim to the visible platform, whereas those with neurotoxic damage to the dorsolateral striatum mainly swim to the invisible platform.

Figure 3

Spatial sensitivity of striatal inhibition. Intracellular recording (left panel) from one VS spiny projection neuron (SPN) in an anesthetized rat showing overlaid responses to tetanic electrical stimulation (arrows) in two different regions (S1, S2) of medial prefrontal cortex. Current injection into the neuron produces tonic firing (gray traces), which is inhibited by stimulation in one site (S1) and enhanced by stimulation in another (S2). The latency of the inhibitory component, its reversal potential (not shown), and data from other studies (see text) indicate that feed-forward inhibition from fast spiking (FS) striatal interneurons in VS and DMS is a likely source of this inhibition (right panel). Abbreviations are the same as for Figure 1. Adapted from Gruber et al. (2009b).

Figure 4

Simple conceptual model of cortico-limbic processing in different scenarios. (A) Presentation of a conditioned stimulus (CS+) early in learning evokes activity in amygdala, thereby triggering VSc activity and elevated dopamine (blue arrows) to engage approach and invigorate strategic actions in DMS. Following reward, direct pathway circuits undergo LTP via dopamine effects on D1 receptors. Many repetitions of such responding will eventually lead to habitual responding in tasks that can be solved by this response strategy. (B) Dopamine levels drop when expected rewards fail to occur, thus increasing sensitivity of striatal SPN to cortical input so as to alter response strategy. LTP ensues in the associated indirect DLS pathway to start the process of conditioned inhibition for that response, while LTD occurs on the associated DLS, DMS, and VS circuits that engaged the unrewarded action. These processes increase the likelihood of selecting a new action in the future. (C) Presentation of a Pavlovian CS engages a PIT mechanism involving additional activation of BLA, VSs, and dopamine to potentiate the associated S-R response in DLS as well as the general vigor via VSc. (D) Presentation of a CS with no associated reward in an excitatory context causes hippocampus to activate inhibition in the VSc so as to prevent orientation and task engagement. Furthermore, if the animal had previously responded to the CS- and received less reward than expected, LTP in the inhibitory indirect pathway would actively suppress these specific responses. Abbreviations are the same as for Figure 1, except that connectivity within the hippocampal formation (HF) is not illustrated.