Covert skill learning in a cortical-basal ganglia circuit

Abstract

We learn complex skills such as speech and dance through a gradual process of trial and error. Cortical-basal ganglia circuits have an important yet unresolved function in this trial-and-error skill learning; influential 'actor-critic' models propose that basal ganglia circuits generate a variety of behaviours during training and learn to implement the successful behaviours in their repertoire. Here we show that the anterior forebrain pathway (AFP), a cortical-basal ganglia circuit, contributes to skill learning even when it does not contribute to such 'exploratory' variation in behavioural performance during training. Blocking the output of the AFP while training Bengalese finches to modify their songs prevented the gradual improvement that normally occurs in this complex skill during training. However, unblocking the output of the AFP after training caused an immediate transition from naive performance to excellent performance, indicating that the AFP covertly gained the ability to implement learned skill performance without contributing to skill practice. In contrast, inactivating the output nucleus of the AFP during training completely prevented learning, indicating that learning requires activity within the AFP during training. Our results suggest a revised model of skill learning: basal ganglia circuits can monitor the consequences of behavioural variation produced by other brain regions and then direct those brain regions to implement more successful behaviours. The ability of the AFP to identify successful performances generated by other brain regions indicates that basal ganglia circuits receive a detailed efference copy of premotor activity in those regions. The capacity of the AFP to implement successful performances that were initially produced by other brain regions indicates precise functional connections between basal ganglia circuits and the motor regions that directly control performance.

Figure 1. Trial-and-error learning in adult birdsong

a. Spectrogram of song during an experiment in which white noise (WN) was delivered to targeted syllable (A) renditions with low fundamental frequency (FF) but not high FF. b. Delivering WN to syllables with low FF (shaded region) elicited increases in FF. Each point corresponds to one syllable rendition; black line indicates running average. c. The song circuit includes a motor pathway, containing HVC and RA, and the anterior forebrain pathway (AFP), important for learning. The AFP generates variation in performance (motor exploration); red and light blue indicate distinct activity patterns in the AFP that lead to distinct FF values on different renditions of the same syllable. d. Actor-critic models propose that the AFP receives feedback about the behavioral variants that it generates, and this feedback strengthens patterns of AFP activity yielding better outcomes (light blue, feedback shown) and weakens patterns of AFP activity yielding worse outcomes (red). e. This changes the output of the AFP so that it selectively implements more successful behaviors. f. We tested this model by blocking the output of the AFP during training, thus preventing the AFP from generating variation in FF. g. The model predicts that this will prevent learning-related plasticity in the AFP, and thus there will be no change in FF, even when AFP output is unblocked after training.

Figure 2. Infusing APV into RA reversibly reduced song variability without distorting song structure

a. The AFP contains the striatopallidal nucleus Area X, the thalamic nucleus DLM, and the cortical nucleus LMAN, which projects to RA. We blocked AFP output to the motor pathway by infusing the NMDA receptor antagonist APV into RA. b. Infusing APV into RA did not markedly change song. c. Infusions of APV into RA reduced the coefficient of variation (CV) of FF, which recovered after switching back to ACSF (n=12 syllables in 9 birds). The CV reduction with APV in RA (31.7% +/− 5.6%) was not significantly different from previously reported effects of lesions (34.1 +/− 4.5%) and inactivations (28.4 +/− 6.0%) of LMAN in adult Bengalese finches. Error bars indicate +/− s.e.m. *Previously reported values from Hampton et al. and Warren et al..

Figure 3. Infusing APV into RA prevents expression but not acquisition of learning

a. Control experiment (ACSF in RA) in which white noise was delivered to targeted syllables with low FF. Arrowheads indicate FF at end of training (1) and after training (2). Dashed line indicates delay between measurements at the end of training and after training. b. For control experiments (n=14 experiments in 7 birds), learning was expressed at a similar magnitude at the end of training (left) and after training (right). Learning was normalized as a percentage of baseline FF. Error bars indicate +/− s.e.m. c. Example of experiment with AFP output blocked (APV infused into RA) throughout the training period. Arrowheads indicate FF at end of training (1) and after training and APV washout (2). d. For experiments with APV in RA (n=21 experiments in 9 birds), learning at end of training (left) was not significantly greater than zero and was significantly less than in control experiments. Learning after training and APV washout (right) was significantly greater than zero and was the same magnitude as in control experiments. e. After training and APV washout, learning was evident in syllables targeted with reinforcement (left) but not in other syllables of the same songs that were not targeted with reinforcement (right). This analysis was performed for each experiment in which FF of a non-targeted syllable could be reliably quantified (n=17 of 21 total experiments). f. Mean progression of learning for control experiments (left) and after unblocking AFP output for experiments with APV in RA (right). Points correspond to syllable renditions 1–5, 1–50, 51–100,…451–500. Dashed lines indicate +/− s.e.m.

Figure 4. Inactivating LMAN during training prevents both expression and acquisition of learning

a. We inactivated LMAN by infusing the GABA_A antagonist muscimol (n=12 experiments in 3 birds) or the sodium channel blocker lidocaine (n=2 experiments in 1 bird) into LMAN. b. Control experiment in which white noise was delivered to renditions of a targeted syllable with low FF. c. Same as panel b, but with LMAN inactivated during training. Arrowheads indicate FF at the end of training with LMAN inactivated (1) and following training and muscimol washout (2). d. For experiments with LMAN inactivated (n=14), there was neither evidence for learning at the end of training (red) nor after training and drug washout (light blue). Error bars indicate +/− s.e.m.