Reward from bugs to bipeds: a comparative approach to understanding how reward circuits function

Abstract

In a complex environment, animals learn from their responses to stimuli and events. Appropriate response to reward and punishment can promote survival, reproduction and increase evolutionary fitness. Interestingly, the neural processes underlying these responses are remarkably similar across phyla. In all species, dopamine is central to encoding reward and directing motivated behaviors, however, a comprehensive understanding of how circuits encode reward and direct motivated behaviors is still lacking. In part, this is a result of the sheer diversity of neurons, the heterogeneity of their responses and the complexity of neural circuits within which they are found. We argue that general features of reward circuitry are common across model organisms, and thus principles learned from invertebrate model organisms can inform research across species. In particular, we discuss circuit motifs that appear to be functionally equivalent from flies to primates. We argue that a comparative approach to studying and understanding reward circuit function provides a more comprehensive understanding of reward circuitry, and informs disorders that affect the brain's reward circuitry.

Keywords: Behavior; dopamine; functional conservation; neuromodulators; neuropeptides; neurotransmitters.

Figure 1.

Reward circuitry across species: schematic of rostral view of fly brain (A) and midsaggital views of zebrafish (B), rodent (C), and primate (D) brains. For clarity only some of the projections are shown and only the main neurotransmitter is represented within each brain region (co-expression is not shown). A legend for neurotransmitter color code is included: green, acetylcholine; purple, dopamine; gray, GABA; blue, glutamate; pink, octopamine/noradrenaline; red, serotonin. Dotted lines represent presumed connectivity. Abbreviations: APL: anterior paired lateral neuron; CA: calyx; CC: cerebellum; CRE: crepine neuropil; DDC: dopaminergic diencephalic cluster; DPM: dorsal paired medial neuron; DRN: dorsal raphe nucleus; IPN: interpeduncular nucleus; LHb: lateral habenula; LH: lateral horn; LDT: laterodorsal tegmentum; LC: locus coeruleus; MB: mushroom bodies; MO: medulla oblongata; NAc: nucleus accumbens; OB: olfactory bulb; OT: optic tectum; PAM: dorsomedial anterior protocerebral; PFC: prefrontal cortex; PPT: pedunculopontine tegmentum; RN: raphe nucleus; rRC: rostral raphe complex; RMTg: rostromedial tegmental nucleus; SC: spinal cord; SMP: superior medial protocerebrum; SP: subpallium; SRN: superior reticular nucleus; Tel: telecephalon; Vd: dorsal nucleus of ventral telencephalic area; VTA: ventral tegmental area; Vv: ventral nucleus of ventral telencephalic area.

Figure 2.

Drosophila mushroom body (MB) innervation patterns. The Drosophila MB comprise Kenyon cell axons that are segregated into three anatomically distinct lobes, the α/β, α/β, and γ outlined in gray) and is compartmentalized based on its innervation pattern. A. Dopamine–acetylcholine circuits. B. Dopamine–GABA circuits. C. Dopamine–glutamate circuits. Subsets of dopamine neurons (left) innervate the same MB compartments as output neurons (middle) and many of these neurons extend axons to the same anatomical regions where the dendrites of dopaminergic neurons are found creating opportunity for putative feedback circuits (right). Abbreviations: CRE: crepine neuropil; LH: lateral horn; PAM: dorsomedial anterior protocerebral dopamine cluster; PPL1: posterior inferiorlateral protocerebrum dopamine cluster; SIP: superior intermediate protocerebrum; SLP: superior lateral protocerebrum; SMP: superior medial protocerebrum.

Figure 3.

Dopamine–noradrenergic circuits implicated in reward and aversion. Top: dopamine–noradrenergic reward. In Drosophila octopamine-dependent sugar reward memory is mediated by OAMB located on dopaminergic neurons. This circuit includes the dopamine PAM cluster, the OA-VPM 3–5 neurons and requires the α-adrenergic-like OAMB receptor (Burke et al., 2012). Similarly, in mammals, noradrenergic projections to dopamine neurons are essential for ethanol reward. In this LC-VTA circuit, noradrenergic LC projection neurons synapse on VTA dopamine neurons (Shelkar et al. 2015). It is thought that this circuit is bidirectional (not shown). Bottom: dopamine–noradrenergic Aversion. In Drosophila, a presumed dopamine–octopamine circuit includes the APL neuron, which co-expresses GABA and octopamine and innervations the MB underlies aversive memory. It is unclear whether APL directly synapses on PAM neurons, however, both of these neurons innervate the MB (Wu et al., 2013). In mammals, noradrenergic projections have also been implicated in processing aversive stimuli. In the dorsal portion of the bed nucleus stria terminalis (BNST) aversive stimuli activate noradrenergic signaling, but inhibits dopaminergic signaling in the ventral BNST (the reverse is true for appetitive stimuli; Park et al. 2011). It is unclear whether noradrenergic and dopaminergic projections interact directly in the BNST. Gray neurons represent unidentified neurons that likely contribute to this reward micro-circuit. Abbreviations: DA: dopamine; NA: noradrenaline/octopamine.

Figure 4.

Dopamine–glutamate circuits implicated in reward and aversion. Top: dopamine–glutamate reward. In Drosophila, interactions between a subset of dopamine PAM cluster neurons and the MB α1 glutamate output neurons underlie sugar reward memory (Ichinose et al., 2015). These glutamatergic neurons extend axons to the superior intermediate protocerebrum (SIP) and superior lateral protocerebrum (SLP) regions where dendrites of the PAM neurons are found. A similar reward circuit was identified in rodents, which include glutamatergic input from the LDT to the VTA (Lammel, Ion, Roeper & Malenka, 2012). These targeted VTA dopaminergic neurons project to the lateral shell of the NAc (not shown). Bottom: dopamine–glutamate aversion. In Drosophila, activation of the M4-6 neurons, which include the MBγ5β′2 glutamatergic output neurons induced odor-driven avoidance (Owald et al., 2015). Interestingly, in a separate study activation of the PAM dopaminergic neurons innervating this MB compartment reduced innate CO₂ avoidance (Lewis et al., 2015). A similar aversion circuit was identified in rodents, which include glutamatergic input from the lateral habenula to the VTA (Lammel et al., 2012). These subsets of VTA neurons project to the prefrontal cortex and RmTg (not shown). Activation of these neurons induced conditioned place avoidance. These circuits are presumed to also be present in primates, however, have yet to be confirmed as indicated by the dotted line. Gray neurons represent unidentified neurons that likely contribute to this aversion micro-circuit. Abbreviations: DA: dopamine; Glu: glutamate.

Figure 5.

Similar neuron complexity across species. (A) Example serotonin neuron in fly brain with discrete projections to the mushroom body and superior dorsofrontal protocerebrum (Chiang et al., 2011). (B) Example contralaterally projecting serotonin-immunoreactive deutocerebral (CSD) neuron. This neuron has broad and extensive innervation pattern in the fly brain. Its dendrites (not shown) innervate the antennal lobe in one hemisphere and its axons project to the lateral horn (LH), superior lateral protocerebrum (SLP), and the contralateral antennal lobe. The CSD neuron has extensive arborizations in these regions as depicted by the areas shaded red (Chiang et al., 2011). (C) Example serotonin neuron recently reconstructed in the mouse brain with discrete projections to the substantia nigra (SN) and the subthalamic nuclei (STN) (Gagnon & Parent, 2014). (D) Example serotonin neuron recently reconstructed in the mouse brain with extensive arborizations in the prefrontal cortex and the hippocampus as depicted by the areas shaded in red (Gagnon & Parent, 2014).