A real-time state predictor in motor control: study of saccadic eye movements during unseen reaching movements

Abstract

Theoretical motor control predicts that because of delays in sensorimotor pathways, a neural system should exist in the brain that uses efferent copy of commands to the arm, sensory feedback, and an internal model of the dynamics of the arm to predict the future state of the hand (i.e., a forward model). We tested this theory under the hypothesis that saccadic eye movements, tracking an unseen reaching movement, would reflect the output of this state predictor. We found that in unperturbed reaching movements, saccade occurrence at any time t consistently provided an unbiased estimate of hand position at t + 196 msec. To investigate the behavior of this predictor during feedback error control, we applied 50 msec random-force perturbations to the moving hand. Saccades showed a sharp inhibition at 100 msec after perturbation. At approximately 170 msec, there was a sharp increase in saccade probabilities. These postperturbation saccades were an unbiased estimator of hand position at saccade time t + 150 msec. The ability of the brain to guide saccades to the future position of the hand failed when a force field unexpectedly changed the dynamics of the hand immediately after perturbation. The behavior of the eyes suggested that during reaching movements, the brain computes an estimate of future hand position based on an internal model that relies on real-time proprioceptive feedback. When an error occurs in reaching movements, the estimate of future hand position is recomputed. The saccade inhibition period that follows the hand perturbation may indicate the length of time it takes for this computation to take place.

Fig. 1.

Experimental setup. A, Subjects used a bite bar and were instructed to try to track their unseen hand during reaching movement. They were prevented from viewing their hands and arms, because both arms were covered under a heavy cloth (data not shown). The right hand held the handle of a robotic arm and moved it in the horizontal plane. The handle housed a high-intensity LED at its center and a force transducer at its base. The hand was below an opaque screen. An LCD projector, held from the ceiling, painted this screen.B, The task began with the robot bringing the hand to a random starting position, where a start target (green square) was displayed. The subject fixated the handle LED. The LED was turned off, and a movement target was displayed (yellow square). The subject saccaded to the target. The movement target was turned off, the handle LED was turned on, and the movement target was redisplayed. The subject fixated the LED. A stationary random-dot pattern was displayed and the start target was removed, signaling the subject to start the reaching movement. As soon as movement was detected, the handle LED was turned off. At the completion of the reach, the handle LED was turned on, the random-dot pattern was removed, and the target square was repainted, providing feedback to the subject. C, Two example trials. The component of eye and hand position parallel to the direction of target is plotted with green and black lines, respectively. Saccade origin is marked with a red dot, and saccade end point is marked with a blue dot.disp., Displacement.

Fig. 2.

Saccades during reaching movements in unperturbed trials. A, Probability (prob.) of saccade occurrence calculated in 10 msec time bins as a function of hand movement time. Each bin indicates the probability that in a single trial, a saccade would occur at that time bin. Reaches were to various target directions. B, Mean trajectory of the reach (± 1 SD; yellow line) is shown by the black line and is represented along directions parallel or perpendicular to the target direction. Saccades are represented by vectors of eye position change: this vector has an origin (red dots) and an end point (blue dots).C, Average timing, origins, and end points of the first three saccades (red and blue lines; ± 1 SD). The SD on timing of saccade end points is identical to the SD of saccade origin and is not shown for clarity. The green line is the average eye position across all trials and corresponds to the continuous representation of the discrete saccade data. Black lines show average hand position. A saccade end point is occasionally not equal to the origin of the subsequent saccade because of small amounts of smooth pursuit.Dir., Direction.

Fig. 3.

Saccades during reaching movements in unperturbed trials. A, Percentage of reaching movements that accompanied a given number of saccades. B, Probability (prob.) of saccade occurrence calculated in 10 msec time bins for reaching movements that had two, three, or four saccades.

Fig. 4.

Saccades during reaching movements in unperturbed trials. A, Eye position at saccade end point and hand position are represented as scalar quantitiese(t) andh(t), indicating position along the direction of the target. The figure illustrates the distribution of the time delays, Δ, for which the error measuree(t) −h(t + Δ) had an average value of zero across all saccades that took place in a 10 msec time bin. Each 10 msec time bin is considered independently, and the corresponding Δ is found. The Δ clusters at 196 ± 31 msec, indicating thate(t) was an unbiased estimator ofh(t + Δ) at Δ ≈196 msec.B, The SD of the error measuree(t) −h(t + Δ) at the Δ for which the measure had zero mean. Time of saccade refers to the time at which the saccade ended.

Fig. 5.

Saccades during reaching movements in unperturbed trials. Eye position at saccade end point and hand position are represented as scalar quantitiese(t) andh(t), indicating position along the direction of the target. The figure illustratesR² between eye position at saccade end point e(t) and hand position at time of saccade t + Δ [(i.e.,h(t + Δ)] for all saccades. The maximum R² is at Δ = 150 msec. Time of saccade refers to the time at which the saccade ended.

Fig. 6.

Saccades during reaching movements in unperturbed trials. Eye position at saccade end point and hand position were represented as two-dimensional quantities indicating position in a Cartesian coordinate system with respect to origin of the reaching movement. The figure illustrates the position of the end point of a saccade and hand position in the corresponding reaching movement at saccade time + 196 msec for all saccades in all trials. Time of saccade refers to the time at which the saccade ended. There is a cluster of points at zero because a number of reaching movements were vertical or horizontal and had no x- or y-components.h, Hand position; e, eye position; m, slope of the line; b, crossing point.

Fig. 7.

Two representative trials during which the hand was perturbed with a force pulse. The gray andblack lines represent the trajectories of eye and hand, respectively. Gray dots locate the end point of each saccade. Black dots indicate hand position at the time of origination of that saccade. For example, e1 is the end point of the first saccade and h1 is the position of the hand at the start time of that saccade. pos., Position; sac, saccade.

Fig. 8.

Trials in which movements were perturbed with a force pulse. A, Saccade probability as a function of movement time. The first gray bar indicates the period (50 msec) during which the reach was perturbed with a force pulse. In all subplots, the second bar is positioned at perturbation (pert.) onset + 150 msec and is 50 msec wide. This period consistently coincides with a peak in postperturbation saccade probabilities. B, Eye positions at the saccade end point for saccades that occurred 150–200 msec after onset of the pulse were represented as two-dimensional variables and were multiplied by a matrix [a −b; b a] to best estimate hand position at t + Δ with respect to the time, t, of the saccade in the same trial. Parameters a andb are shown (±95% confidence interval). Whena = 1 and b = 0, the saccade vector remains unscaled and unrotated. A rotation, arctan(b/a), and scaling, $\sqrt{(a^2+b^2)}$, of the saccade vector was needed to estimate hand position at all delays except at Δ = 150 msec. For this time lead, the actual saccades were an unbiased estimator of hand position. C, Force pulses were perpendicular to the direction of the target and displaced the hand by various amounts in each movement. The end points of saccades that occurred 150–200 msec after the pulse are plotted versus hand position in the same movement at saccade time + 150 msec. perp. dir, Perpendicular direction;m, slope of the line; b, crossing point.

Fig. 9.

In these trials, a viscous force field that either assisted or resisted the movement of the hand was introduced immediately after the offset of the force pulse to the hand.A, Saccade probabilities. The first gray bar indicates pulse period (50 msec). The second gray bar indicates a perturbation (pert.) period 150–200 msec after pulse onset. B, The end points of saccades that occurred 150–200 msec after the pulse are plotted versus hand position in the same movement at saccade time + 150 msec. Saccades overestimated hand position in resistive trials and underestimated hand position in assistive trials. perp. dir, Perpendicular direction.