On-line processing of uncertain information in visuomotor control

Abstract

Our sensory observations represent a delayed, noisy estimate of the environment. Delay causes instability and noise causes uncertainty. To deal with these problems, theory suggests that the processing of sensory information by the brain should be probabilistic: to start a movement or to alter it midflight, our brain should make predictions about the near future of sensory states and then continuously integrate the delayed sensory measures with predictions to form an estimate of the current state. To test the predictions of this theory, we asked participants to reach to the center of a blurry target. With increased uncertainty about the target, reach reaction times increased. Occasionally, we changed the position of the target or its blurriness during the reach. We found that the motor response to a given second target was influenced by the uncertainty about the first target. The specific trajectories of motor responses were consistent with predictions of a "minimum variance" state estimator. That is, the motor output that the brain programmed to start a reaching movement or correct it midflight was a continuous combination of two streams of information: a stream that predicted the near future of the state of the environment and a stream that provided a delayed measurement of that state.

Figure 1.

Experimental procedures and results from the control 1 experiment. A, Subjects held a handle that housed an LED (red mark) and fixated a cross at 10 cm with respect to the start box. The reach target was a fuzzy blob that appeared at a random position and at a random time (0.5–2.5 s) after the presentation of the fixation cross. Subjects were instructed to start their reach as soon as they saw the target and terminate their reach by placing the LED in the center of the blob. The LED was always on. After reach completion, feedback was provided to indicate distance to the center of the blob. B, On a fraction of trials, at ∼100 ms after reach initiation, the center of the blob and/or SD changed. C, Endpoint SDs and reaction times (RT) are shown as a function of blob SD in the task shown in A. As the target uncertainty increased, reaction times increased, and movements ended with greater spatial variance with respect to target center. Data points are mean and SEM across subjects.

Figure 2.

The computational problem of estimating the state of the environment despite delay and noise in the sensory measurements. A, A “graphical model” representation of the delay and noise problems. Each circle depicts a random variable. The shaded circles are variables that are observed, and the unshaded circles are unobserved variables that must be estimated. The arrows describe conditional probabilities, or simply causality. The vector x represents information about state of the hand (position and velocity) and the target (its center). Observation y^(t) is a measure of the delayed state x_Δ^(t). Given this observation, we need to estimate the current state x₀^(t) and then produce motor commands u such that the hand arrives in the center of the target. B, A simplified example of the estimation problem. The observation suddenly changes at time t. x_Δ^{(t|t − δ)} is the prior estimate before observation y^(t) and x̂_Δ^(t|t) is the posterior estimate after the observation. With repeated observations, the mean of the estimate converges onto the observation. C, Simulations of a point mass system controlled by a time-delayed controller. Top row, The target jumps at t = 0. This change is observed after a time delay Δ = 100 ms. After the system observes the change in the target position, the estimated target position gradually converges to the observed target position. However, the speed of the convergence is affected by both the uncertainty of the prior (position of the target before it jumped) and the uncertainty of the observation (position of the target after it jumped). If the first target had a small uncertainty with respect to the second uncertainty, then the convergence rate is slow (e.g., S–M condition or M–L condition). If the first target had a large uncertainty with respect to the second uncertainty, then the convergence rate is fast (L–M and M–S conditions). Middle row, The motor command in response to the target jump. The peak response to the target jump is highest for L–M and M–S conditions. The duration of the response (time at which the curve crosses 0) is smallest for the L–M and M–S conditions. Bottom row, Motion of the simulated hand in response to the target jump. The peak response is higher and earlier in the L–M and M–S conditions. D, The endpoint SD of the simulated trajectories (500 trials in each condition). E, The time evolution of target uncertainty. Left, The uncertainty during the reaction time. Before the first target is displayed, uncertainty is very large. When the first target is displayed, the rate of reduction in the uncertainty is faster for the S target than for M or L targets. If we suppose that the system starts to initiate a reach a constant time after the uncertainty crosses an arbitrary threshold, then reaction times are shorter for S targets than for M or L. Right, The uncertainty during the reach when the target changes. By the time the reach starts, target uncertainty is near asymptotic levels. The uncertainty increases or decreases when the target changes. In the L–M condition, the uncertainty is higher than in the M–M or S–M conditions after the target jump, resulting in a greater reliance in the delayed sensory observations and therefore a faster correction.

Figure 3.

Main experiment. A, Left, The mean hand path of the response induced by target jump. The mean was calculated with data across all subjects. Right, Temporal patterns of y velocity. Data was aligned by the onset of target jump. The mean data for all conditions are shown but are almost precisely overlapping. B, Top, There were no detectable changes in the y accelerations as a function of target conditions. Middle, However, there were clear changes in the motor response along the x-axis (x acceleration) with changing uncertainty of the target. When the first uncertainty was small with respect to the second target, the motor response was large. When the uncertainty of the first target was large with respect to the second target, the motor response was small. Bottom, An enlarged scale of the motor response (x acceleration). The peak and zero-crossing point were delayed when the peak response increased. C, There were main effects associated with target uncertainties in peak response. Top, Rate of initial response (middle) and time of peak response (bottom).

Figure 4.

Control experiment 2. A, Left, Mean hand paths of the response induced by target jump. The mean was calculated with data across all subjects. Right, Temporal patterns of y velocity. B, Top, The response along the y-axis was indistinguishable across various conditions. Bottom, However, changing the target uncertainty produced clear variations in the response along the x-axis. When the uncertainty associated with the first target was small with respect to the second target, the motor response to the second target was large. When the uncertainty associated with the first target was large with respect to the second target, the response was small. The peak and zero-crossing points were delayed when the peak response increased. C, There were main effects associated with target uncertainties in peak response (top), rate of initial response (middle), and time of peak response (bottom).