Discrete Evaluative and Premotor Circuits Enable Vocal Learning in Songbirds

Abstract

Virtuosic motor performance requires the ability to evaluate and modify individual gestures within a complex motor sequence. Where and how the evaluative and premotor circuits operate within the brain to enable such temporally precise learning is poorly understood. Songbirds can learn to modify individual syllables within their complex vocal sequences, providing a system for elucidating the underlying evaluative and premotor circuits. We combined behavioral and optogenetic methods to identify 2 afferents to the ventral tegmental area (VTA) that serve evaluative roles in syllable-specific learning and to establish that downstream cortico-basal ganglia circuits serve a learning role that is only premotor. Furthermore, song performance-contingent optogenetic stimulation of either VTA afferent was sufficient to drive syllable-specific learning, and these learning effects were of opposite valence. Finally, functional, anatomical, and molecular studies support the idea that these evaluative afferents bidirectionally modulate VTA dopamine neurons to enable temporally precise vocal learning.

Keywords: actor-critic; basal ganglia; birdsong; dopamine; optogenetics; reinforcement learning; skill learning; ventral tegmental area; vocal learning; zebra finch.

Figure 1:. Pitch-contingent auditory feedback negatively reinforces syllable pitch.

(A) Spectrogram of zebra finch song highlighting three sequential syllables from a motif. A target syllable (black note), showing the related premotor window (orange) and auditory feedback window (purple). (B) Experimental design for pitch learning, here targeting lower pitch syllable variants (After Tumer and Brainard 2007). (C) Left, pitch of the first and last 50 renditions of a target syllable on a single day at baseline (black). Right, the first and last 50 renditions of a target syllable on a single day where white noise (WN) was delivered on low pitch renditions of the syllable (red). (D) Mean increase in pitch of target syllables over one day of WN learning (n = 15 syllables, n = 15 birds, p=0.00017, paired t-test). (E) Schematic of song learning. (F) Sagittal drawing of zebra finch brain emphasizing some of the nuclei involved in song learning and production.

Figure 2:. Aiv_VTA and VP_VTA convey evaluative information important for vocal learning.

(A) Syllable sequence highlighting premotor and auditory feedback “evaluative” windows associated with syllable B. (B) Low pitch syllable renditions trigger disruptive WN feedback while high pitch renditions “escape” WN. (C) Schematic of premotor optogenetic “interference” experiment. As in Figure 1, pitch-contingent WN is delivered to syllable D. Independently, the preceding syllable C is detected on every rendition and triggers the laser at a brief fixed delay to deliver optogenetic stimulation during the premotor window for 100% of syllable D renditions. (D) Schematic of auditory feedback optogenetic “interference” experiment. The same approach is used as in the premotor jamming experiment, but now the delay for triggering the laser is set to coincide with the auditory feedback window associated with syllable D. (E) Schematic of Aiv_VTA experiments. (F) Change in target syllable pitch across a single day without laser stimulation or WN (baseline (B), black), WN on low pitch variants (No Laser + WN, red), WN with feedback laser (Feedback Laser + WN, purple), and WN with premotor laser (Premotor Laser + WN, orange). (G) Bar graph of percent change in pitch of target syllables following one day in each of the four conditions (No Laser + WN, n=5, p=0.0489, Feedback Laser + WN, n=5, p = 0.2374, Premotor Laser + WN, n=5, p=0.0235, all paired t tests). (H) Schematic VP_VTA experiments. (I) Same as in (F) except for VP_VTA. (J) Same as in (G) except for VP_VTA, (No Laser + WN, n=4, p=0.0308, Feedback Laser + WN, n=4, p=0.4812, Premotor Laser + WN, n =4, p = 0.0077, all paired t tests).

Figure 3:. The cortical output of the BG pathway serves a premotor role important to pitch learning.

(A) Viral targeting and stimulation strategy. (B) Histological confirmation of ChR2 expression (red) in LMAN, scale bar, 50 μm (green anti-Calbindin). (C) High power image of ChR2 expression (red) in LMAN, scale bar 10 μm. (D) 5 trials of single unit activity recorded in LMAN in an isoflurane-anesthetized finch, in response to 20 ms laser pulses (blue triangles). (E) Top, latency to first spike relative to onset of laser stimulation. Bottom, inter-spike interval histogram of LMAN activity evoked by laser (blue) and spontaneous activity (red). (F) Change in target syllable pitch across a single day without laser stimulation or WN (baseline (B), black), WN on low pitch variants (No Laser + WN, red), WN with feedback laser (Feedback Laser + WN, purple), and WN with premotor laser (Premotor Laser + WN, orange). (G) Bar graph of percent change in pitch of target syllables following one day in each of the four conditions (No Laser + WN, n=5, p=0.0353, Feedback Laser + WN, n=5, p=0.0019, Premotor Laser + WN, n=5, p=0.6114, all paired t tests).

Figure 4:. Pitch-contingent stimulation of Aiv_VTA terminals negatively reinforces target syllable pitch.

(A) Viral targeting and stimulation strategy. (B) Top, ChR2 (red) expression in Aiv. Bottom, Aiv terminals in VTA (TH+ cells in green). Scale bars: 50 μm. (C) Top, schematic of experimental design, scale bars: 100 ms, 2 kHz. Left, pitch of all “catch” trials of the target syllable at baseline and on each day of laser stimulation (B, baseline; L1, first day light stimulation, L2, second day light stimulation, etc., renditions below threshold in blue). Right, z-scored pitch distribution of a target syllable before and after stimulation on low pitch renditions. (D) Left, baseline z-scored pitch distributions from all target syllables, each distribution represents one target syllable. Middle, z-scored pitch distribution from target syllables after stimulation on high pitch renditions. Right, as in middle but for stimulation on low pitch renditions. (E) auROC for all target syllables on B1 or L4 compared to B2 (n=6 syllables, 4 birds). (F) Change in frequency (hertz) for two baseline days and four days of VP_VTA stimulation for syllables targeted on low pitches (n = 3, blue, bold line is average across syllables) and high pitch renditions (n = 4, red, bold line is average across syllables). (G) Mean percent change in pitch on B1 and L4, relative to B2, in experimental birds (p=0.0029 paired t test). Right, same but in control birds (n=4 syllables, n=4 birds, p=0.4805, paired t test, green: GFP, gray: uninjected).

Figure 5:. Pitch-contingent activation of VP_VTA terminals positively reinforces target syllable pitch.

(A) Viral targeting and stimulation strategy. (B) Top, ChR2 (red) expression in VP. Bottom, VP terminals in VTA (TH+ cells, green). Scale bars: 50 μm. (C) Top, schematic of experimental design. Left, pitch of the target syllable at baseline and on each day of laser stimulation (convention as in Figure 4). Right, z-scored pitch distribution of a target syllable before and after stimulation on low pitch renditions. (D) Left, baseline z-scored pitch distributions from all target syllables. Each histogram represents one syllable. Middle, z-scored pitch distributions from target syllables after stimulation on high pitch variants. Right, as in middle but for stimulation on low pitch variants. (E) auROC for all target syllables on B1 or L4 compared to B2 (n=7 syllables, 6 birds). (F) Change in frequency (hertz) for two baseline days and four days of VP_VTA stimulation for syllables targeted on low pitches (n=3, blue, bold line is average across syllables) and high pitch renditions (n=4, red, bold line is average across syllables). (G) Mean percent change in pitch on B1 and L4, relative to B2, in experimental birds (n=7 syllables, n=6 birds, p=0.0006 paired t test). Right, same but in control birds (n=4 syllables, n=4 birds, p=0.2017, paired t test, green: GFP, gray: uninjected).

Figure 6:. Optogenetic stimulation of Aiv_VTA and VP_VTA terminals drive bi-directional and opposing effects on VTA neurons.

(A) Schematic showing optogenetic activation of Aiv_VTA terminals while recording extracellularly from VTA neurons. (B) Same as (A), but for VP_VTA terminals. (C) Left, example recording of fast firing ‘thin-spiking’ unit in VTA (pink). Right, spike waveforms, mean waveform in bold and single spikes in gray. (D) Same as (C) but for a slow firing “thick-spiking” unit in VTA (E) Top, raster plot of thin-spiking single unit in VTA. Below, PSTH of activity showing increased firing with Aiv terminal activation (laser, cyan). Right, spike waveform, mean and single spikes shown. Bottom, raster plot of thick-spiking single unit in VTA and below, PSTH of activity showing decreased firing with Aiv terminal activation. Right, spike waveform, mean and single spikes shown. (F) Same as E but for VP_VTA terminals (G) Summary data from n=20 neurons from n=14 birds recorded in the VTA while stimulating Aiv terminals. Each unit that significantly modulated with laser stimulation plotted by spike width and firing rate (+, increased activity with laser; − decreased activity with laser, red region containing putative ‘thin-spiking’ interneurons and pink region containing putative ‘thick-spiking’ dopamine neurons). (H) Same as G but for VP_VTA experiments, data from n=12 neurons from n=8 birds recorded in the VTA while activating VP.

Figure 7:. Neurotransmitter phenotypes and synaptic structure of VTA afferents.

(A) Upper left, schematic of injection of CTB into VTA to label Aiv_VTA neurons. Upper right, Aiv_VTA projecting neurons (magenta), VGLUT2 (green), and VGAT (red), scale bar 50 μm. Bottom left, higher magnification views, all scale bars 10 μm. Bottom right, pie chart of percentage of Aiv_VTA cells positive for VGLUT2, VGAT, both, and neither (gray). (B) Upper left, schematic of injection of CTB into VTA to label VP_VTA neurons. Upper right, VP_VTA projecting neurons (teal), VGLUT2 (green), and VGAT (red), scale bar 50 μm. Bottom left, higher magnification views, all scale bars 10 μm. Bottom right, pie chart of percentage of VP_VTA cells positive for VGLUT2, VGAT, both, and neither. (C) Left, viral injection of GFP into Aiv to express in Aiv_VTA terminals. Top middle and right, PV+ (red) and TH+ neurons (pink) respectively with Aiv terminals (green) in VTA, scale bar 50 μm. Bottom middle and right, higher magnification view showing Aiv terminals in apposition to PV+ (middle, red) and TH+ (right, pink) cell bodies, scale bar 10 μm. (D) Same as (C) but for VP_VTA GFP terminals. (E) Top, bar graph of appositions from Aiv_VTA GFP terminals on PV+ (red) and TH+ (pink) cells in the VTA. Bottom, same as (E) but for VP_VTA GFP terminals onto VTA neurons. (F) Upper Left, schematic of injection of CTB into VTA and VP to retrogradely label Aiv_VTA neurons and Aiv_VP neurons. Bottom left, pie chart of overlap of Aiv_VP neurons with Aiv_VTA neurons. Top right, Aiv_VTA (red), Aiv_VP (green), and co-localized (yellow) neurons, scale bar 200 μm, sagittal view with the song nucleus RA outlined. Bottom right, higher magnification view, scale bar 50 μm.

Figure 8:. Circuit enabling vocal learning in the songbird

(A) Diagram of zebra finch brain with brain nuclei involved in vocal learning. (B) Schematic of circuit for vocal learning, informed by the present study.