Flexible, task-dependent use of sensory feedback to control hand movements

Abstract

We tested whether changing accuracy demands for simple pointing movements leads humans to adjust the feedback control laws that map sensory signals from the moving hand to motor commands. Subjects made repeated pointing movements in a virtual environment to touch a button whose shape varied randomly from trial to trial-between squares, rectangles oriented perpendicular to the movement path, and rectangles oriented parallel to the movement path. Subjects performed the task on a horizontal table but saw the target configuration and a virtual rendering of their pointing finger through a mirror mounted between a monitor and the table. On one-third of trials, the position of the virtual finger was perturbed by ±1 cm either in the movement direction or perpendicular to the movement direction when the finger passed behind an occluder. Subjects corrected quickly for the perturbations despite not consciously noticing them; however, they corrected almost twice as much for perturbations aligned with the narrow dimension of a target than for perturbations aligned with the long dimension. These changes in apparent feedback gain appeared in the kinematic trajectories soon after the time of the perturbations, indicating that they reflect differences in the feedback control law used throughout the duration of movements. The results indicate that the brain adjusts its feedback control law for individual movements "on demand" to fit task demands. Simulations of optimal control laws for a two-joint arm show that accuracy demands alone, coupled with signal-dependent noise, lead to qualitatively the same behavior.

Figure 1.

a, A side view of the experimental apparatus used in the experiments. b, Subjects' view of the tabletop environment (seen from the top) halfway through a trial. Subjects viewed the display stereoscopically through LCD stereo glasses with geometrically correct rendering of the three-dimensional scene. The start position and target rectangle appeared flat in the plane of the tabletop (approximately coextensive with the virtual image of the monitor). The semicircular occluder appeared 10 cm above the tabletop, so that subjects' fingers disappeared as they passed behind the occluder. c, A schematic rendering of a perturbation trial, in which the position of the virtual finger was shifted up by 1 cm (in the plane of the tabletop) relative to the true finger position.

Figure 2.

A schematic view of the two-joint arm model used for the simulations.

Figure 3.

a, The scatter of endpoints on unperturbed trials for a representative subject in experiment 1 for each of the three targets. The coordinate frame is aligned with the axis between starting and target positions. Ellipses represent twice the SD of endpoint positions. b, Average magnitudes of corrections to 1 cm perturbations of the virtual finger for the three button shapes. Corrections shown are in the direction of the perturbation. Perturbations parallel to the axis between the start position and the target button are shown as horizontal arrows; perturbations perpendicular to that axis are shown as horizontal arrows. Error bars indicate SEMs.

Figure 4.

Example velocity profiles for two subjects pointing to the square target. a and c show subjects' average fingertip velocities for three perturbation conditions—no perturbation; positive 1 cm perturbation in the principal direction of movement, parallel to the path between the start position and the target; and negative 1 cm perturbation in the same direction. Velocities shown are the velocity components parallel to the path. b and d show the average differences in velocities for positive and negative perturbations. The gray area represents the SEM difference. The transparent gray rectangles represent the time that subjects' fingers were behind the occluder for the average trajectory. The true occlusion times varied from trial to trial.

Figure 5.

Example velocity profiles for two subjects pointing to the square target. a and c show subjects' average fingertip velocities for three perturbation conditions—no perturbation, positive 1 cm perturbation perpendicular to the path between the start position and the target, and negative 1 cm perturbation perpendicular to the path. Velocities are in the direction perpendicular to the path. b and d show the average differences in perpendicular velocities for positive and negative perturbations. The gray area represents the SEM difference. The transparent gray rectangles represent the time that subjects' fingers were behind the occluder for the average trajectory. The true occlusion times varied from trial to trial.

Figure 6.

Perturbation influence functions calculated for the same subjects and conditions whose kinematics are shown in Figures 5 and 6. SEs shown in gray were calculated by bootstrap. We resampled the trials used to compute each influence function and calculated the SD of the resulting bootstrapped estimates of the influence functions. a, b, Influence functions calculated for horizontal perturbations and square targets from the kinematic data shown in Figure 5. c, d, Influence functions calculated for vertical perturbations and square targets from the kinematic data shown in Figure 6.

Figure 7.

Perturbation influence functions averaged across the eight subjects for the three button shapes. a, Vertical perturbations (perturbations perpendicular to the axis between the start and target positions). b, Horizontal perturbations (perturbations parallel to the axis between the start and target positions). c, d, Same plots for horizontal and vertical rectangle conditions with error bars (SEs of the between subjects' means) plotted in gray. e, f, Average within subject differences in the influence functions shown in c and d with error bars in gray.

Figure 8.

a, Response times estimated the collection of all subjects' influence functions for the six target shape and perturbation direction combinations. Response times were estimated as the point at which the average influence functions exceeded 0 by >2 SEs and remained above that threshold (see, for example, Fig. 5c,d). b, Estimates of the times at which subjects' corrective responses to perturbations for vertical and horizontal rectangles diverged for the two perturbation directions using the same criterion (see Fig. 5e,f). Error bars show SEs estimated from a bootstrap procedure in which response times were estimated using the above procedure for 1000 random draws, with replacement, of the eight influence functions derived for each subject.

Figure 9.

The average magnitude of subjects' perturbation influence functions between 150 and 200 ms after the finger reappeared from behind the occluder. a, Results for vertical perturbations. b, Results for horizontal perturbations. Error bars indicate SE.

Figure 10.

a, Average endpoint corrections for perturbations in experiment 2, expressed as a proportion of the perturbation. Corrections shown here are projections of the two-dimensional correction vectors onto the axis of the perturbation. Error bars indicate SE. b, Two-dimensional plot of subjects' average correction vectors in response to diagonal perturbations as calculated from Equation 1. This should be read as subjects' corrections to a perturbation of the finger by [−0.707, −0.707] (in centimeters). Since responses to positive and negative perturbations are averaged, the negative of these vectors would reflect subjects' average corrections to perturbations of the finger of [0.707, 0.707]. The solid lines show the measured correction vectors. The dashed lines show the corrections vectors predicted by a linear sum of the corrections to vertical and horizontal perturbations. Ellipses centered at the ends of the vectors are SE covariance ellipses.

Figure 11.

Perturbation influence functions for the three button shapes. a, Influence functions computed from the y component of subjects' trajectories in trials with vertical perturbations. b, Influence functions computed from the x component of subjects' trajectories in trials with horizontal perturbations. c, d, Perturbation influence functions derived from the y and x components, respectively, of subjects' trajectories computed from trials containing diagonal perturbations. The dashed curves show the perturbations influence functions predicted from a linear superposition of the perturbation influence functions computed from the trials with vertical and horizontal perturbations (a, b).

Figure 12.

The scatter of endpoints on unperturbed trials for a representative subject in experiment 2 for each of the three targets. The coordinate frame is aligned with the axis between starting and target positions.

Figure 13.

a, Endpoint covariance ellipses for optimal controllers derived for each of the three target shapes individually. Ellipses represent twice the SD of endpoint positions. b, The average magnitude of correction for 1 cm perturbations in horizontal and vertical directions for each of the three controllers (the controller is indexed by the target shape shown on the x-axis).

Figure 14.

Perturbation influence functions computed for the optimal controllers derived for each shape. a, Results for vertical perturbations. b, Results for horizontal perturbations.