Neural Encoding and Representation of Time for Sensorimotor Control and Learning

Abstract

The ability to perceive and produce movements in the real world with precise timing is critical for survival in animals, including humans. However, research on sensorimotor timing has rarely considered the tight interrelation between perception, action, and cognition. In this review, we present new evidence from behavioral, computational, and neural studies in humans and nonhuman primates, suggesting a pivotal link between sensorimotor control and temporal processing, as well as describing new theoretical frameworks regarding timing in perception and action. We first discuss the link between movement coordination and interval-based timing by addressing how motor training develops accurate spatiotemporal patterns in behavior and influences the perception of temporal intervals. We then discuss how motor expertise results from establishing task-relevant neural manifolds in sensorimotor cortical areas and how the geometry and dynamics of these manifolds help reduce timing variability. We also highlight how neural dynamics in sensorimotor areas are involved in beat-based timing. These lines of research aim to extend our understanding of how timing arises from and contributes to perceptual-motor behaviors in complex environments to seamlessly interact with other cognitive processes.

Keywords: beat based timing; dynamic systems; interval based timing; motor timing; sensorimotor control; temporal processing.

Figure 1.

A dynamic systems view on brain and behavior in the context of perception, action, and cognition. This perspective challenges the theoretical framework of a centralized clocking mechanism by showing how temporal processing in perception and sensorimotor actions is achieved by coordinating perceptual, motor, and cognitive processes.

Figure 2.

Simplified virtual throwing task. A, In the virtual task, the participant performs forearm movements via a manipulandum and throws a virtual ball to hit a target on the screen. The error is defined and calculated as the shortest distance that the ball trajectory achieves to the target. The time at which this closest distance occurs can differ between trials. B, The task has redundancy as infinitely many different ball releases (angle and velocity at ball release) can achieve a given error. The combinations of angle and velocity that achieve zero error define the solution manifold (green band). Orange line indicates an exemplary arm trajectory plotted in the same space. It closely aligns with the solution manifold where a “timing window” can be defined. Any ball release within this window achieves a zero-error target hit (Zhang et al., 2018). C, Continuous arm movements plotted in phase space spanned by position and velocity, display a closed orbit, indicating periodicity. With practice, successive throws develop a stable periodic pattern (from day 1 to day 4). Black dots indicate the ball releases. The variability of these trajectories significantly decreases from day 1 to day 4. Red line indicates a Poincare section, where the intersections of the arm trajectory are analyzed to test for stability (Zhang and Sternad, 2019). D, Velocity profile of two successive throwing movements illustrates different temporal intervals defined by kinematic landmarks. Red dot indicates the ball release time. The interval between the start of the movement to the ball release (∼300 ms) is most critical and positively affects interval perception.

Figure 3.

Representation of time during time interval reproduction and rhythmic timing. A, Top, Time interval production task. Monkeys were required to estimate a sample interval demarcated by Ready and Set, and reproduce that interval by a delayed motor response (Go). Sample intervals were drawn from one of two prior distributions: Short or Long. Bottom, A schematic showing the curved neural trajectory during the Ready-Set epoch for the Short prior condition. Linear readout of time intervals from the curved neural trajectory (left) generates biased internal estimates of the sample interval (middle) and reduces variability near the extrema of the prior distribution (right). B, Top, Synchronization task. Monkeys were required to tap (circles) synchronously three intervals (SO1-SO3) to an external metronome (arrows). The interstimulus interval was either Short or Long. Bottom, Neural trajectories during the synchronization task. The trajectory starts from a tapping manifold (black line), completes a cycle during every intertap interval, and returns to the tapping manifold. The tapping manifold is invariant across durations and serial order elements of the task. The metronome's tempo modulates the amplitude of the trajectories and the serial order element as the third axes in the state population.