Motor primitives in space and time via targeted gain modulation in cortical networks

Abstract

Motor cortex (M1) exhibits a rich repertoire of neuronal activities to support the generation of complex movements. Although recent neuronal-network models capture many qualitative aspects of M1 dynamics, they can generate only a few distinct movements. Additionally, it is unclear how M1 efficiently controls movements over a wide range of shapes and speeds. We demonstrate that modulation of neuronal input-output gains in recurrent neuronal-network models with a fixed architecture can dramatically reorganize neuronal activity and thus downstream muscle outputs. Consistent with the observation of diffuse neuromodulatory projections to M1, a relatively small number of modulatory control units provide sufficient flexibility to adjust high-dimensional network activity using a simple reward-based learning rule. Furthermore, it is possible to assemble novel movements from previously learned primitives, and one can separately change movement speed while preserving movement shape. Our results provide a new perspective on the role of modulatory systems in controlling recurrent cortical activity.

Figure 1. Controlling network activity through neuron-specific gain modulation.

(a) Example of a reaching task, with illustrative electromyograms (EMGs) of muscle activity for two reaches (in orange and black). (b) Schematic of our model (see the text and Methods Section 1.10). (c) (Left) Changing the slope of the input–output gain function uniformly for all neurons from (black) 1 to (blue) 2 has pronounced effects on (right) neuronal firing rates. We show results for three example neurons. (d) The mean error in network output decreases during training with neuron-specific modulation. In the inset, we show five snapshots of network output (indicated by arrowheads) as learning progresses. (e) (Left) Neuronal gain changes during training for 2 example neurons (grey and black) and 10 training sessions to the same target. (Right) Histogram of gain values after training. The blue curve is a Gaussian fit with a standard deviation of σ ≈ 0.157. (f) Network outputs (grey curves) with all gains set to 1 and a new learned gain pattern for 10 noisy initial conditions compared to both targets (black and orange). (We use a 200-neuron network for all simulations in this figure.)

Figure 2. Learning through gain modulation in different models.

(a) Mean error over 10 independent training sessions for our original model that we used in Fig. 1d (red); the model with a biologically motivated ramping input (blue); the model when using the alternative learning rule Eqn. (10), in which learning automatically stops at a sufficiently small error (purple); and when using a ‘chaotic’ recurrent network model (grey) (see Methods Section 1.10). Shading indicates one standard deviation. (b) The firing rates of 4 example neurons before (i.e., with all gains set to 1) and after training the neuronal gains in (left) our original model, (centre left) our model with a ramping input, (centre right) our model with the alternative learning rule, and (right) the model when using a ‘chaotic’ network. (We use 200-neuron networks for all simulations in this figure.)

Figure 3. Controlling network activity through coarse, group-based gain modulation.

(a) We identically modulate neurons within each group (see Methods Section 1.9). Target outputs can involve multiple readout units. (b) Mean error during training for 20 random, 20 specialized, and 200 (i.e., neuron-specific) groups. (See Methods Section 1.10 for more details.) (c) Mean minimum errors after training using specialized groups. We use the same grouping for learning multiple different movements. (d) Mean minimum errors for different numbers of random groups with networks of 100, 200, and 400 neurons. (The N on the horizontal axis indicates neuron-specific modulation.) In panels (b)–(d), we use a single readout unit. (e) (Top) Mean minimum error as a function of the number of random groups when learning each of (left) 2, (centre) 3, and (right) 4 readouts for the same networks as in panel (d). (Bottom) The corresponding mean errors during training for the case of 40 groups. The inset is a magnification of the initial training period for the case of 2 readout units. (f) Outputs producing the median error for the case of 4 readout units using 40 groups in the 400-neuron network.

Figure 4. Gain patterns can provide motor primitives for novel movements.

(a) Schematic of a learned library of gain patterns (g₁,…,g_l, which we colour from purple to blue) and a combination c₁F(g₁) +…+ c_lF(g_l) of their outputs (which we denote by F) that we fit (red dashed curve) to a novel target (grey curve). (Upper right) The output F(c₁g₁ +…+ c_lg_l) (which we show in orange) of the same combination of corresponding gain patterns also closely resembles the target. We use a 400-neuron network with 40 random modulatory groups (see Methods Section 1.10). (b) Example target, fit, and output (grey, red dashed, and orange curves, respectively) producing the 50th-smallest output error over 100 randomly generated combinations (see Methods Section 1.10) of l library elements using l = 2, l = 4, l = 8, and l = 16. (c) Fit error versus output error for 100 randomly generated combinations of l library elements for l = 1,…, 20. We show the identity line in grey. Each point represents the 50th-smallest error between the output and the fit across 100 novel target movements. (d)Median errors of the 100 randomly-generated combinations of l library elements versus the number of library elements.

Figure 5. Examining effects of more strongly nonlinear neuronal dynamics by using a baseline rate of r₀ = 5 Hz.

(a) Relative firing rate of 20 excitatory and 20 inhibitory neurons in a 200-neuron network with r₀ = 20 Hz in Eqn. (2). (b) Relative firing rate of the same neurons as those in panel (a), but with r₀ = 5 Hz. (c) The dotted curves show the relative firing rates of all neurons over time when using the nonlinear gain function (see Eqn. (2)) with (black) r₀ = 20 Hz and (blue) r₀ = 5 Hz versus the relative firing rates that result from using the linear gain function f(x_i; g_i) = g_ix_i. We set each neuronal gain g_i to 1, and we plot the identity line in grey. (d) Mean error over 10 independent training sessions with r₀ = 20 Hz (black) and with r₀ = 5 Hz (blue) for the task in Fig. 1d (see Methods Section 1.10). Shading indicates one standard deviation. In the inset, we show network outputs with all gains set to 1 and the new learned gain pattern with r₀ = 5 Hz for 10 noisy initial conditions (grey curves). We show the two targets in black and orange (see Methods Section 1.10). (e) Histogram of gain values after training with r₀ = 5 Hz. The black curve is a Gaussian distribution with a mean of 1 and a standard deviation of σ ≈ 0.157 (i.e., the distribution that we obtained with r₀ = 20 Hz in Fig. 1e). (f) Gain patterns as motor primitives with r₀ = 5 Hz. We generate these results in the same manner as our results in Fig. 4d, except that now we use r₀ = 5 Hz. We obtain qualitatively similar results to our observations for the baseline rate r₀ = 20 Hz.

Figure 6. Gain modulation can control movement speed.

(a) Schematic of gain patterns for fast (0.5 s) and slow (2.5 s) movement variants. (Here and throughout the figure, we show the former in blue and the latter in orange.) We train a 400-neuron network using 40 random modulatory groups for all simulations (see Methods Section 1.10). (b) (Top) We train a network to extend its output from a fast to a slow-movement variant using our reward-based learning rule. (Bottom) Example firing rates of 50 excitatory and 50 inhibitory neurons for both fast and slow speed variants. (c) The same as panel (b), but now we use a back-propagation algorithm to train the neuronal gains (see Methods Section 1.10). (d) (Top) Interpolation between fast and slow gain patterns does not reliably generate target outputs of intermediate speeds when trained only at the fast and slow speeds. We show an example output (orange) that lasts a duration of 1.5 s and the associated target (grey). (Bottom) Linear interpolation between the fast and slow gain patterns successfully generates target outputs when trained at 5 intermediate speeds. We train 1 set of gain patterns (see panel (e)) on two target outputs associated with 2 different initial conditions (see Methods Section 1.10). (We plot these results with the same axis scale as in the top panel.) (e) The 7 optimized gain patterns for all 40 modulatory groups when training at 7 evenly-spaced speeds. (f) Both outputs when linearly interpolating at 5 evenly-spaced speeds between the fast and slow gain patterns from panel (e).

Figure 7. Joint control of movement shape and speed through gain modulation.

(a) One can jointly learn the gain patterns ${\mathit{\text{g}}}_i^s$ for (left box) movement speed and ${\mathit{\text{g}}}_j^m$ for (right box) movement shape so that the product of two such gain patterns produces a desired movement at a desired speed. In the rightmost panel, we show example outputs for two movement shapes at 3 interpolated speeds between the fast and slow gain patterns. (See the main text.) (b) We show the 7 optimized gain patterns for controlling movement speed (i.e., ${\mathit{\text{g}}}_i^s$ for i ∈ {1,…,7} from panel (a)) for the 40 modulatory groups when training on 10 different movement shapes. (c) We plot the mean error over all 10 movements when linearly interpolating between the fast and slow gain patterns for controlling movement speed from panel (b). We use the same vertical axis scale as in Fig. 6d. In the inset, we plot the same data using a different vertical axis scale. The vertical dashed lines identify the 7 movement durations that we use for training. (d) Outputs at 5 interpolated speeds between the fast and slow gain patterns for 6 of the 10 movements. (For each simulation, we train a 400-neuron network using 40 random modulatory groups (see Methods Section 1.10).)

Figure 8. Learning gain-pattern primitives to control movement shape and speed.

(a) We are able to learn to combine (left) previously acquired gain patterns for movement shapes to generate (centre) a new target movement at both fast and slow speeds simultaneously using (right) a fixed manifold in neuronal gain space for controlling movement speed (see Methods Section 1.10). (b) We plot the output, at 3 different speeds, that produces the 50th-smallest error (across all 100 target movements) between the output and the target when summing errors at both fast and slow speeds. (c) Mean network output error across all 100 target movements for all durations when learning to combine gain patterns (black solid curve). We plot the error for the output from panel (b) in red. As a control, we plot the mean error over all target movements when dissociating the learned gain patterns from their target movement by permuting (uniformly at random) the target movements (see the grey curve). We also plot the mean error over all target movements when combining gain patterns using a least-squares fit of the 10 learned movement shapes to the target (black dashed curve) (see Methods Section 1.10). (For each example, to generate outputs of a specific duration, we linearly interpolate between the fast and slow gain patterns.) (d) We plot the Pearson correlation coefficient between each pair of target movements versus the Pearson correlation coefficient between the corresponding pair of learned combination coefficients$c_1\text{,}\text{…}\text{,}{c}_{10}$.