Temporal Evolution of Spatial Computations for Visuomotor Control

Abstract

Goal-directed reaching movements are guided by visual feedback from both target and hand. The classical view is that the brain extracts information about target and hand positions from a visual scene, calculates a difference vector between them, and uses this estimate to control the movement. Here we show that during fast feedback control, this computation is not immediate, but evolves dynamically over time. Immediately after a change in the visual scene, the motor system generates independent responses to the errors in hand and target location. Only about 200 ms later, the changes in target and hand positions are combined appropriately in the response, slowly converging to the true difference vector. Therefore, our results provide evidence for the temporal evolution of spatial computations in the human visuomotor system, in which the accurate difference vector computation is first estimated by a fast approximation.

Keywords: motor control; neural computations; visual processing; visuomotor.

Figure 1.

Methods and responses to isolated target or cursor perturbations. A, Subjects made reaching movements while grasping a planar robotic manipulandum. Visual feedback of targets and hand position (cursor) were provided in the plane of the reach via a mirror. B, On random probe trials, the cursor, the target, or both were perturbed laterally by one of seven distances for 250 ms while the physical hand position was constrained by a mechanical channel to the straight line between the start and final target. Subjects fixated visual gaze to the fixation cross (yellow), which signaled the start of movement cue by changing color. The presentation side (right or left) for the fixation cross varied from block to block in the experiment. C, The combination of seven possible target perturbations (green) and seven possible cursor perturbations (red) gave rise to 49 separate conditions where the diagonals had identical difference vectors. D, Left, Mean force responses (solid line) and SEM (shaded region) across all subjects to isolated target displacements plotted with respect to the time of displacement onset. The vertical dotted line indicates the mean (±1 SD) of the response onset time. The mean force responses were averaged over an early interval (170–230 ms, dark gray bar, middle) and a late interval (370–430 ms, light gray bar, right) for comparison across conditions. Error bars indicate ±1 SEM. The mean force responses were fit with a logistic function to the target responses of each time interval separately. E, Mean force responses to isolated cursor displacements analog to D.

Figure 2.

Models and exemplary predictions. A, The difference vector model combines estimates of target and hand positions into a single difference vector, which is sent to the motor system for corrective responses. B, The weighted difference vector allows different weightings between target and hand estimate. C, The multichannel model considers two separate feedback pathways through the visual and motor systems. The force response produced by each model depends on any nonlinearity in the motor system that can be estimated using the isolated target and cursor responses. D, The theoretical output of each of the models for a simple logistic function representing the relationship between displacement size and corrective response. The colors indicate the size and direction of the corrective forces. The black dashed line indicates the diagonal in which the difference vector is −1 cm. E, Theoretical predicted force responses for the −1 cm difference vector diagonal for the difference vector (blue), weighted difference vector (green), and multichannel (red) model.

Figure 3.

Responses to conditions with equal difference vectors. A, Left, Conditions with +2 cm difference vector. Middle left, Mean force responses (solid line) and SEM (shaded region) across all subjects. Middle right, The mean (±1 SEM) force responses over the early interval (dark gray shaded time interval, 170–230 ms). The horizontal lines indicate statistical differences (A, E, p < 0.005; and B–D, p < 0.0033; Bonferroni-corrected for multiple comparisons) between conditions that have been selected to demonstrate that the responses are nonmonotonic, and the corresponding arrows indicate the direction of the difference. Colored lines represent the predicted mean force responses from each of the three models: difference vector (blue), weighted difference vector (green), and multichannel model (red). Note that these predictions are based only on the responses to the isolated target and cursor perturbations. Right, The force responses over the later interval (light gray shaded time interval, 370–430 ms). B, Difference vector of +1 cm. C, Difference vector of 0 cm. D, Difference vector of −1 cm. E, Difference vector of −2 cm.

Figure 4.

Responses to isolated target or cursor perturbations for a single subject. A, Left, Mean force responses (solid line) and SEM (shaded region) across all trials for one exemplar subject to isolated target displacements plotted with respect to the time of displacement onset. The mean force responses were averaged over an early interval (170–230 ms, dark gray bar, middle) and a late interval (370–430 ms, light gray bar, right) for comparison across conditions. Error bars indicate ±1 SEM. B, Mean force responses to isolated cursor displacements.

Figure 5.

Responses to conditions with equal difference vectors for a single subject. A, Left, Conditions with +2 cm difference vector. Middle left, Mean force responses (solid line) and SEM (shaded region) across all trials for the same exemplar subject as Figure 4. Middle right, The mean (±1 SEM) force responses over the early interval (dark gray shaded time interval, 170–230 ms). Right, The force responses over the later interval (light gray shaded time interval, 370–430 ms). B, Difference vector of +1 cm. C, Difference vector of 0 cm. D, Difference vector of −1 cm. E, Difference vector of −2 cm.

Figure 6.

Time course of model fits. A, Predicted variance for the 36 combined conditions for each model as a function of the time from perturbation onset (±SEM). The black line indicates the noise ceiling, i.e., the maximal predictable variance given the intrasubject variance throughout the time. Shaded regions indicate the early and late time period from Figures 1 and 3. The p values above those quantify the comparison difference vector versus the multichannel model (two-sided t tests) B, Force responses during the early time window (170–230 ms after perturbation onset) plotted across all cursor and target displacements. Top left, Experimental data. Top right, Predicted responses of the difference vector model. Bottom left, Predicted responses from the multichannel model. Bottom right, Predicted responses from the weighted difference vector model. Note that each participant's equal-difference vector diagonals were monotonously increasing; only their average partially displays a nonmonotonous trend. Models are fitted to each subject's average data from isolated target and cursor perturbations, and the extrapolated predictions are shown for combined perturbations. C, Force responses during the late time window (370–430 ms after perturbation onset) and model predictions.

Figure 7.

Time course of model fits based on velocity data on the trials in which subjects were free to move. A, Predicted variance for the 36 combined conditions for each model as a function of the time from perturbation onset (±SEM). The black line indicates the noise ceiling, i.e., the maximal predictable variance given the intrasubject variance throughout the time. The proportion of variance predicted developed differentially for the three models over the time course of movement (interaction model by time, F_(24,456) = 29.417, p < .001). Shaded regions indicate the early and late time periods. The p values above those quantify the comparison difference vector versus the multichannel model (two-sided t tests). B, Velocity responses during the early time window (170–230 ms after perturbation onset) plotted across all cursor and target displacements. Top left, Experimental data. Top right, Predicted responses of the difference vector model. Bottom left, Predicted responses from the multichannel model. Bottom right, Predicted responses from the weighted difference vector model. Note that each participant's equal-difference vector diagonals were monotonously increasing; only their average partially displays a nonmonotonous trend. Models are fitted to each subject's average data from isolated target and cursor perturbations, and the extrapolated predictions are shown for combined perturbations. C, Velocity responses during the late time window (370–430 ms after perturbation onset) and model predictions.