The role of feedback in the production of skilled finger sequences

Abstract

Actions involving fine control of the hand, for example, grasping an object, rely heavily on sensory information from the fingertips. Although the integration of feedback during the execution of individual movements is well understood, less is known about the use of sensory feedback in the control of skilled movement sequences. To address this gap, we trained participants to produce sequences of finger movements on a keyboard-like device over a 4-day training period. Participants received haptic, visual, and auditory feedback indicating the occurrence of each finger press. We then either transiently delayed or advanced the feedback for a single press by a small amount of time (30 or 60 ms). We observed that participants rapidly adjusted their ongoing finger press by either accelerating or prolonging the ongoing press, in accordance with the direction of the perturbation. Furthermore, we could show that this rapid behavioral modulation was driven by haptic feedback. Although these feedback-driven adjustments reduced in size with practice, they were still clearly present at the end of training. In contrast to the directionally specific effect we observed on the perturbed press, a feedback perturbation resulted in a delayed onset of the subsequent presses irrespective of perturbation direction or feedback modality. This observation is consistent with a hierarchical organization of even very skilled and fast movement sequences, with different levels reacting distinctly to sensory perturbations.NEW & NOTEWORTHY Sensory feedback is important during the execution of a movement. However, little is known about how sensory feedback is used during the production of movement sequences. Here, we show two distinct feedback processes in the execution of fast finger movement sequences. By transiently delaying or advancing the feedback of a single press within a sequence, we observed a directionally specific effect on the perturbed press and a directionally non-specific effect on the subsequent presses.

Keywords: finger movements; haptic feedback; motor learning; movement sequences; sensory feedback.

Graphical abstract

Figure 1.

Two hypothetical representations of sensory feedback in fast motor sequences. A: a hierarchical controller that represents the movement sequence across multiple interacting layers. The sequence controller represents the sequences of finger presses and activates the corresponding finger controllers, which are then responsible for generating the muscle commands for moving the finger. Sensory feedback (dashed line) goes back both to the relevant finger controller and to the overall sequence controller. B: after training, neighboring finger presses may be represented as a movement chunk, which generates the muscle commands for multiple, overlapping finger presses. Sensory feedback would lead to slowing or acceleration of the execution of the entire movement chunk.

Figure 2.

Apparatus and achieved time advancements of feedback. A: isometric keyboard-like device. Each key was associated with a number (these numbers were not shown to the participants but verbally explained). B: histogram of the time intervals between feedback presentation and the actual press onset for the two advancement conditions. Vertical doted lines indicate −30 and −60 ms. The delay conditions could always be achieved accurately.

Figure 3.

Calculation of feedback differences across presses and landmarks. A: for our analyses we calculated time intervals between the onset of the perturbed press (blue onset dot in the figure) and different force landmarks (green dots) on the perturbed press as well as on subsequent presses (indicated with +1). We chose five specific time landmarks on each press: early onset (≥0.75 N), onset (≥1.5 N), peak (maximum force between onset and release), release (first time <1.5 N after onset), and late release (first time <0.75 after onset). B: for the calculation of the offset, we choose a single time point (onset of +1 press) and found the corresponding force level on the perturbed press for unperturbed trials (black line). For perturbed trials (red line), we then located the same landmarks. The offset is defined as the difference between the estimated delay on the next press (interval 1) and the estimated simultaneous delay on the perturbed press (interval 2).

Figure 4.

Effects of perturbation on perturbed and subsequent press. A and C: average force traces for day 1 and day 4 and the following press interpolated and standardized to the average time of each condition. Dotted line indicates press onset, for which the sensory feedback was shifted in time. Error bars represent the mean ± standard error across subjects. B and D: differences between the onset-to-peak and onset-to-release intervals of perturbed and unperturbed trials for day 1 and day 4.

Figure 5.

Effects of feedback perturbation on the perturbed press (press 0) and subsequent finger presses (Press +1, Press +2) across feedback conditions and training days. Five landmarks (EO: early onset, O: onset, P: peak, R: release, LR: later release) are plotted per press (see methods). The x axis shows the average time of occurrence of the landmark on unperturbed trials relative to the onset of the first press. The y axis shows the time difference of the landmarks between the perturbation conditions and the unperturbed condition. Landmarks belonging to a finger press are connected by a line. Positive differences indicate that the perturbation resulted in a delay, whereas negative differences indicate a speed-up. The four panels show results for the different training sessions (i.e. days). Day 4 shows how we tested the offset between presses, with an example of the 2nd to 3rd press for the +60 ms condition. Error bars represent the means ± standard error across participants.

Figure 6.

Dependence of offset and overlap between presses on day 4. Dots show the data for each individual participant for each of the six specific presses that could be perturbed. The x axis shows the time from the release of the perturbed to the onset of the next press. Smaller values indicate more overlap. The y axis shows the offset measured in the +60 ms perturbation condition (see Fig. 5).

Figure 7.

Effect of haptic, visual, and auditory feedback perturbation in control experiment across training days. As in Fig. 5, five landmarks per press (connected by a line) are plotted. The control experiment only had +80 ms perturbations, but each group of subjects received only one type of feedback. The different panels indicate the different training sessions (i.e. days). The error bars represent the means ± standard error across participants for each group.