The Avian Basal Ganglia Are a Source of Rapid Behavioral Variation That Enables Vocal Motor Exploration

Abstract

The basal ganglia (BG) participate in aspects of reinforcement learning that require evaluation and selection of motor programs associated with improved performance. However, whether the BG additionally contribute to behavioral variation ("motor exploration") that forms the substrate for such learning remains unclear. In songbirds, a tractable system for studying BG-dependent skill learning, a role for the BG in generating exploratory variability, has been challenged by the finding that lesions of Area X, the song-specific component of the BG, have no lasting effects on several forms of vocal variability that have been studied. Here we demonstrate that lesions of Area X in adult male zebra finches (Taeniopygia gutatta) permanently eliminate rapid within-syllable variation in fundamental frequency (FF), which can act as motor exploration to enable reinforcement-driven song learning. In addition, we found that this within-syllable variation is elevated in juveniles and in adults singing alone, conditions that have been linked to enhanced song plasticity and elevated neural variability in Area X. Consistent with a model that variability is relayed from Area X, via its cortical target, the lateral magnocellular nucleus of the anterior nidopallium (LMAN), to influence song motor circuitry, we found that lesions of LMAN also eliminate within-syllable variability. Moreover, we found that electrical perturbation of LMAN can drive fluctuations in FF that mimic naturally occurring within-syllable variability. Together, these results demonstrate that the BG are a central source of rapid behavioral variation that can serve as motor exploration for vocal learning.SIGNIFICANCE STATEMENT Many complex motor skills, such as speech, are not innately programmed but are learned gradually through trial and error. Learning involves generating exploratory variability in action ("motor exploration") and evaluating subsequent performance to acquire motor programs that lead to improved performance. Although it is well established that the basal ganglia (BG) process signals relating to action evaluation and selection, whether and how the BG promote exploratory motor variability remain unclear. We investigated this question in songbirds, which learn to produce complex vocalizations through trial and error. In contrast with previous studies that did not find effects of BG lesions on vocal motor variability, we demonstrate that the BG are an essential source of rapid behavioral variation linked to vocal learning.

Keywords: basal ganglia; motor exploration; reinforcement learning; social context; songbird; vocal learning.

Figure 1.

Song variability at multiple timescales and underlying neural circuits. A, Variation in song acoustic structure on different timescales. Top, A spectrogram illustrating four motifs of an example song consisting of syllables a–e (squares indicate motifs). In many previous studies, “cross-rendition variability” in FF was used to examine song variability (left). This variability is obtained for a particular syllable (d in this example) by computing mean FF of the flat harmonic portion of individual syllable renditions (dotted squares, middle) and by measuring variations in this feature across renditions (bottom left). In contrast, the present study focuses on “within-syllable variability” (right), the rapid fluctuations of FF trajectories within individual syllable renditions (blue trace on the spectrogram that is an expanded view of the square area in the syllable on the left). B–D, Schematic representations of the effects of lesions in the AFP on song variability. The neural circuit underlying vocal learning and production (B) and magnitudes of cross-rendition (C) and of within-syllable variability (D) are shown. In the circuit diagrams, red crosses indicate lesions; yellow, blue, and green boxes indicate BG, thalamic, and cortical structures, respectively. Str, Striatum; GPi, internal segment of the globus pallidus. HVC is used as a proper name, and arrowheads and filled circles indicate excitatory and inhibitory neural projections, respectively. Note that lesions of LMAN (middle) have been shown to dramatically decrease cross-rendition variability compared with that in intact birds (left), whereas lesions of Area X (right) do not have sustained effects on cross-rendition variability. The effects of Area X and LMAN lesions on within-syllable variability have not been previously examined (question mark in the diagram) and are the focus of this study.

Figure 2.

Subtle effects of Area X lesions on acoustic and temporal structure in adult song motifs. A, Spectrograms illustrate a variable degree of changes in the motif structure of Area X-lesion birds. For each bird, a motif of pre-lesion (Pre) and 8 week (8wk) post-lesion (Post 8wk) song are shown. Arrows indicate apparent changes in syllable structure. B, C, Summary of spectral similarity (B) and temporal similarity (C) between pre-lesion and 8 week post-lesion song motifs (see Materials and Methods). Some of these data were published previously (Kojima et al., 2013) and are replotted here with new data. Each point corresponds to one bird; birds 1–3 shown in A are indicated with arrows. Note that although the majority of the birds maintained the motif structure as much as control birds, a subset of birds showed noticeable changes in both spectral and temporal structure of their song motifs. D, Area X lesions lowered mean FF of song syllables slightly. Percent changes in mean FF in individual syllables (averaged across renditions) from pre-lesion song to post-8 week song are shown. Each point corresponds to one syllable. *p < 0.02. E, Area X lesions shortened song motifs, as reported previously (Kubikova et al., 2014). Pre–post percent changes in motif duration (averaged across renditions) are shown. Each point corresponds to one bird. *p < 0.02.

Figure 3.

Area X is required for within-syllable vocal variability. A, Circuit diagram indicating lesions of Area X (left) and spectrograms of songs of an adult zebra finch recorded before and after 2 week (2wk) bilateral lesions of Area X (right). Conventions are as in Figure 1, A and B. B, Examples of FF trajectories of syllable e in pre-lesion (left) and 2 week post-lesion (right) songs, expressed as raw frequency traces (top), percent deviation from cross-rendition mean (middle), and percent deviation from within-rendition mean (bottom). Twenty consecutive trajectories are shown. Red traces indicate across-rendition means. C, The magnitude of within-syllable FF variability (within-rendition SD of mean-subtracted FF trajectory, averaged across renditions) normalized by that of pre-lesion song in Area X-lesion (red) and control (black) birds. Thin lines correspond to one syllable, and thick lines indicate the mean across all syllables. **p < 10⁻⁴. D, Power spectra of FF trajectories pre-lesion (blue) and 2 weeks post-lesion (magenta) in control (top) and Area X-lesion (bottom) birds, normalized by the peak height of the spectrum in pre-lesion song; each line corresponds to one syllable. E, Changes in mean (±SEM) power spectrum of FF trajectories from pre-lesion to 2 weeks post-lesion, normalized to the peak height of the presong spectrum, in Area X-lesion (red) and control (black) birds. The red bars indicate the frequency ranges where Area X-lesion data were significantly different from control data (see Materials and Methods). F, For the same syllables, Area X lesions did not permanently eliminate cross-rendition variability in FF. Conventions are as in C. *p < 0.01.

Figure 4.

Social context modulation of within-syllable variability depends on Area X. A, Examples of 20 consecutive FF trajectories of a single syllable produced by an intact bird in Undir and Dir contexts show context-dependent modulation of within-syllable FF variability. B, Scatter plots of the magnitude of within-syllable FF variability comparing Undir context with Dir context. Each point corresponds to one syllable. The dashed line indicates unity. p < 10⁻⁴. C, Comparison of the effects of social context versus lesions of Area X on within-syllable FF variability. Green, Changes (mean ± SEM) in the power spectrum of FF trajectories from Undir to Dir contexts in intact birds, normalized to the peak height of Undir song spectrum; red, pre–post changes in the power spectrum of Undir song in Area X-lesion birds (data from Fig. 3E replotted for comparison). Conventions are as in Figure 3E. Note the absence of significant differences between Undir–Dir data and Area X-lesion data. D, Within-syllable FF variability in Dir and Undir contexts in control (left) and Area X-lesion (right) birds; variability at all time points and in both contexts were normalized by that of Dir syllables in pre-lesion song. Error bars are SEM. *p < 10⁻³; **p < 10⁻⁴. E, Cross-rendition FF variability in Dir and Undir contexts in control (left) and Area X-lesion (right) birds. Conventions are as in D.

Figure 5.

LMAN is required for within-syllable variability. A, Circuit diagram indicating lesions of LMAN. B, Within-syllable FF variability in LMAN-lesion (red) and control (black) birds in Undir context; conventions are as in Figure 3C. **p < 10⁻⁵. C, Pre–post changes in the power spectrum of FF trajectories in LMAN-lesion (red) and control (black) birds in Undir context. Conventions are as in Figure 3E. D, Within-syllable FF variability in Dir and Undir contexts in LMAN-lesion birds. Conventions are as in Figure 4D. *p < 0.02; **p < 10⁻³.

Figure 6.

Microstimulation in LMAN can elicit rapid deflections of FF trajectories in both intact birds and birds with Area X lesions. A, Circuit diagram showing electrical microstimulation in LMAN of an intact bird. B, Example of a syllable with (right; stim) and without (left; no-stim) LMAN stimulation in an intact bird. Stimulation (10 ms duration) is indicated by the black bar. C, Interleaved FF trajectories in stim (red) and no-stim (blue) trials for the syllable shown in B, normalized to the cross-rendition mean of no-stim trajectories. Dashed lines in yellow and green indicate the mean of FF trajectories in stim trials and 2 SDs of FF trajectories in no-stim trials, respectively; the thick bar indicates the timing of LMAN stimulation; and the arrow indicates the onset of FF deflections (see Materials and Methods). D, Power spectra of FF trajectories with and without LMAN stimulation for the syllable shown in B and C. Black lines indicate the median in each condition. E, Percent difference in power between median power spectra with and without stimulation (mean ± SEM; n = 11 syllables from 8 intact birds in total; colors indicate durations of LMAN stimulation (10, 20, and 50 ms). As a comparison, percent changes in the power spectrum of naturally occurring FF fluctuations after Area X lesions are also plotted (gray lines; the same data as in Fig. 3E; the direction of the difference in power is opposite between LMAN stimulation data and Area X lesion data because LMAN stimulations increase power spectra and Area X lesions decrease power spectra). Note that stimulation-driven deflections of FF trajectories have a timescale comparable with that of Area X-dependent fluctuations. F–J, The same experiment in birds with Area X lesions (n = 6 syllables from 4 birds). Conventions are as in A–E. K, Examples of two syllables in which LMAN stimulations disrupted the harmonic structure. L, Peak frequencies of stimulation-dependent power spectra in individual syllables of intact (green) and lesion (magenta) birds. No significant differences in peak frequencies were observed for all the stimulation durations (p = 0.11, 0.24, and 0.81 for 10, 20, and 50 ms, respectively). M, Latencies of FF deflections in intact (green) and lesion (magenta) birds. No significant differences were observed for all the stimulation durations (p = 0.53, 0.69, and 0.21, respectively).

Figure 7.

Within-syllable FF variability decreases with age. A, Changes in the magnitude of within-syllable FF variability over 2 weeks in juvenile (left) and adult (right) birds. Each line corresponds to one syllable. *p < 0.002. B, Percent changes (mean ± SEM) in the power spectrum of FF trajectories in juvenile (blue) and adult (black) birds over the 2 week period shown in A. Note the greater reduction of power in juvenile birds compared with that in adult birds.