Prospective errors determine motor learning

Abstract

Diverse features of motor learning have been reported by numerous studies, but no single theoretical framework concurrently accounts for these features. Here, we propose a model for motor learning to explain these features in a unified way by extending a motor primitive framework. The model assumes that the recruitment pattern of motor primitives is determined by the predicted movement error of an upcoming movement (prospective error). To validate this idea, we perform a behavioural experiment to examine the model's novel prediction: after experiencing an environment in which the movement error is more easily predictable, subsequent motor learning should become faster. The experimental results support our prediction, suggesting that the prospective error might be encoded in the motor primitives. Furthermore, we demonstrate that this model has a strong explanatory power to reproduce a wide variety of motor-learning-related phenomena that have been separately explained by different computational models.

Figure 1. Model schematic.

(a) A schematic representation of our model. Prospective error determines activities of motor primitives, the weighted sum of these activities determines motor commands and movement error is a learning signal for the weighting parameters and prospective error. (b) Force field: subjects need to move the cursor or his/her hand towards the target. (c) Visuomotor rotation: subjects need to move the cursor towards the target.

Figure 2. Structural learning.

We investigated whether our model could explain the entire structural learning process. Each p_t was randomly sampled from the subset s=(−45°, −30°, −15°, 0°, 15°, 30°, 45°) in the training trials. In groups 1, 2 and 3, the perturbation sequence varied in every trial, every two trials and every three trials, respectively. Washout trials were inserted between the training and test trials. These washout trials excluded the possibility that the movement error in the last training trial affects the learning speed in the test trials. During the test phase, a constant visuomotor rotation, p=(30°, ⋯, 30°), was imposed. (a) Trial-by-trial change of x_t (thick line) and p_t (thin line) in group 2. (b) Activity of each primitive in group 2, where a strong white colour indicates high activity. The vertical axis shows the sorted preferred prospective error, μ_i, from −90° to 90°. The red line denotes the prospective error. (c) Weighting parameters of each primitive in group 2. Blue and red colours indicate weighting parameters to compensate for perturbations of positive and negative values, respectively. The red dotted line shows that motor primitives for a 30° PE learn the 30° perturbation in the training trials. (d) Comparison of x_t in the adaptation to the visuomotor rotation among the three groups. Each x_t value is calculated by averaging across 100 simulations. (e) Activities of each primitive in group 1. (f) Weighting parameters of each primitive in group 1. The green dotted line shows that motor primitives for a 30° PE do not learn the 30° perturbation in the training trials.

Figure 3. Schematic diagram of the behavioural experiment.

(a) Subjects needed to adapt to a −30° or 30° visuomotor rotation after experiencing the force channel trials (see below). (b) Throughout the experiment, the target direction was fixed to 90°. In the force channel trials, the actual hand-movement direction was also fixed to 90° using a virtual wall (force channel trial). In groups 1, 2 and 3, the cursor movement varied randomly in every trial, every two trials and every three trials, respectively. (c) Prediction of our model in test trials (each x_t value is calculated by averaging across 100 simulations). In this simulation, x_t in each force channel trial is forcibly set to 0. The PE can be predicted more reliably in groups 2 and 3 than in group 1, and the motor learning is predicted to be facilitated in groups 2 and 3 compared with group 1: learning speed is significantly higher in groups 2 and 3 than in group 1.

Figure 4. Results of our behavioural experiment.

(a) Generated force at the (t+1)-th trial after experiencing a movement error e_t in the force channel trials in group 1 (black dotted line, mean±s.e.m., n=12). The green solid line shows the fitting of our model (R²=0.9950). (b) Actual data (mean±s.e.m., n=12 for each group) and learning curves predicted by our model (R²=0.8638 for group 1 (green), R²=0.7967 for group 2 (red) and R²=0.7968 for group 3 (blue)). Notably, the parameters were fit to data from only group 1, and our model predicted the learning curves for groups 2 and 3 with these parameters. Data for the adaptation to the 30° and −30° visuomotor rotations are included in each group. (c) Histogram of bootstrapped learning speed. Vertical solid lines denote the mean values of each distribution.

Figure 5. Model fitting to data in crcns.org.

(a) Data from Körding and Wolpert. Solid lines show the fit of our model (R² was 0.9315, 0.9448, 0.9823 and 0.9786 for data of σ₀, σ_M, σ_L and σ_∞, respectively). Dotted lines show actual data (mean±s.e.m., n=10). (b) Data from Wei and Körding. Solid line shows the fit of our model (R² was 0.8947). Dotted line shows actual data (mean±s.e.m., n=7). (c) Data from Thoroughman and Taylor. Solid line shows the fit of our model (R² was 0.8240). Dotted line shows moving average filtered actual data (mean, n=12).

Figure 6. Uncertainty effect.

To determine whether our model can explain an uncertainty effect, we simulated an experiment in which the model adapts to a 30° visual rotation for 50 trials with an observation noise, that is, e_t=p_t−x_t+ξ_t, where ξ_t is a Gaussian random noise with a mean of 0 and a variance of . When σ_G is large, uncertainty is large for the observation of the movement error. (a) Trial-by-trial change of x_t averaged across 100 simulations. (b) Adaptation rate after fitting a state-space model x_t+1=Ax_t−Be_t to the simulated x_t shown in a, where A is a forgetting rate and B is an adaptation rate. (c) Previously reported adaptation rate (reproduced from Wei and Körding4).

Figure 7. Savings.

(a–d) To examine our model’s ability to explain the short-term savings effect, we considered an experiment in which a 30° visual rotation was applied for 30 trials (the initial learning phase), followed by a −30° visuomotor rotation for 5 trials (the opposite learning phase), and another set of the 30° visual rotation for 30 trials (the relearning phase). (a) Trial-by-trial change of p_t and x_t averaged across 10 simulations of short-term savings. (b) The activity of each primitive, with a strong white colour indicating high activity. The red line denotes the prospective error, ê_t. Vertical dotted lines are drawn at the trials when the phases switched. The horizontal dotted line denotes the line on which ê_t=0. (c) Weighting parameters of each primitive. Blue and red colours indicate weighting parameters to compensate for perturbations of positive and negative values, respectively. (d) Comparison of x_t between the initial learning and relearning phases. (e) Persistence of the savings effect and its dependence on the forgetting rate (λ=0.9586 (best-fit parameter for the data of group 1), 9786 or 9986). We simulated an experiment in which a 30° visual rotation was applied for 60 trials (the initial learning phase) followed by a 0° visuomotor rotation (washout phase), and another set of the 30° visual rotation was imposed for 20 trials (the relearning phase). The horizontal axis denotes the length of the washout trials. (Inset) comparison of x_t between the initial learning and relearning phases. We define the savings effect as the integral of the grey zone: the difference of x_t in the first five trials between the initial learning and relearning phases. This value should be 0 if there are no savings, and the value is positive when the learning speed in the relearning phase is higher than that in the initial learning phase. The savings effects were normalized by setting the maximal value to be 1. (f) Previously reported savings by Krakauer et al. (adapted by permission from Macmillan Publishers Ltd: Nature Neuroscience, copyright 1999).

Figure 8. Anterograde interference.

To determine whether our model could explain this effect, we simulated an experiment in which a 30° visual rotation for 50 trials (the initial learning phase) was followed by a −30° visuomotor rotation for 50 trials (the opposite learning phase). (a) Trial-by-trial change of p_t and x_t averaged across 10 simulations. (b) Activities of each primitive, with a strong white colour indicating high activity. The red line denotes the prospective error, ê_t. Vertical dotted line is drawn at the trial at which the initial learning phase switches to the opposite learning phase. The horizontal dotted line denotes the line on which ê_t=0. (c) Weighting parameters of each primitive. Blue and red colours indicate weighting parameters to compensate for perturbations of positive and negative values, respectively. (d) Comparison of x_t between the initial learning and opposite learning phases. In the opposite learning phase, the negative part of x_t is drawn (red line in a). (e) Trial-by-trial change of x_t in the opposite learning phase. Each dotted line denotes the dependence of x_t on the length of the initial learning phase. (f) Previously reported savings by Sing and Smiath (reproduced from a previous study22).

Figure 9. Spontaneous recovery.

We simulated an experiment in which a 30° visual rotation for 50 trials (the initial learning phase) was followed by a −30° visuomotor rotation for 5 trials (the opposite force-learning phase), and error-clamp trials were imposed. In the simulation of the error-clamp trials, the movement error, e_t, was forcibly set to 0°. (a) Trial-by-trial change of p_t and x_t averaged across 10 simulations. (b) Previously reported spontaneous recovery (reproduced from Smith et al.1). (c) Activities of each primitive, with a strong white colour indicating high activity. Vertical dotted lines are drawn for the trials when the phases switched. Horizontal dotted line denotes the line on which ê_t=0. (d) Weighting parameters for each primitive. Blue and red colours indicate weighting parameters to compensate for perturbations of positive and negative values, respectively.

Figure 10. Comparison of the prospective error model and perturbation prediction model.

(a) Trial-by-trial change of p_t and x_t averaged across 10 simulations. The grey zone denotes error-clamp trials in which the error, e_t, was forcibly set to 0°. (b) Activities of each primitive in the perturbation prediction model when is forcibly set to 0° in error-clamp trials, with a strong white colour indicating high activity. Red line denotes predicted perturbation. Vertical dotted lines are drawn for the trials when the phases switched. The horizontal dotted line denotes the line on which . (c) Weighting parameters of each primitive when is forcibly set to 0° in error-clamp trials. Blue and red colours indicate weighting parameters to compensate for perturbations of positive and negative values, respectively. (d) Activities of each primitive in the perturbation prediction model when is forcibly set to −30° in error-clamp trials. (e) Weighting parameters of each primitive when is set to −30° in error-clamp trials.