Multiple grasp-specific representations of tool dynamics mediate skillful manipulation

Abstract

Skillful tool use requires knowledge of the dynamic properties of tools in order to specify the mapping between applied force and tool motion. Importantly, this mapping depends on the orientation of the tool in the hand. Here we investigate the representation of dynamics during skillful manipulation of a tool that can be grasped at different orientations. We ask whether the motor system uses a single general representation of dynamics for all grasp contexts or whether it uses multiple grasp-specific representations. Using a novel robotic interface, subjects rotated a virtual tool whose orientation relative to the hand could be varied. Subjects could immediately anticipate the force direction for each orientation of the tool based on its visual geometry, and, with experience, they learned to parameterize the force magnitude. Surprisingly, this parameterization of force magnitude showed limited generalization when the orientation of the tool changed. Had subjects parameterized a single general representation, full generalization would be expected. Thus, our results suggest that object dynamics are captured by multiple representations, each of which encodes the mapping associated with a specific grasp context. We suggest that the concept of grasp-specific representations may provide a unifying framework for interpreting previous results related to dynamics learning.

Figure 1

Robotic Manipulandum and Virtual Tool Manipulation Task (A) The WristBOT is a planar two-dimensional robotic manipulandum that includes torque control at the vertical handle. Cables and pulleys (two are shown) implement the transmission system between the handle and drive system at the rear of the manipulandum (not shown). A safety cover that encloses the handle pulley and cables has been removed for clarity. (B) Top view of subject showing visual feedback of a virtual tool, which is projected over the subject's hand in the plane of movement. Visual feedback (see C) is consistent with grasping the tool at its base. In reality, subjects grasp the vertical handle of the WristBOT, which is aligned with visual feedback. The WristBOT handle translates in the horizontal plane (x and y) and rotates around the vertical axis. Subjects view visual feedback in a mirror that prevents them from seeing either their hand or the manipulandum. Dotted line shows subject's midsagittal plane, which is aligned with the hand and the vertical rotation axis of the tool. Inset shows top view of subject's hand overlaid with five different visual orientations of the tool. (C) Virtual tool dynamics were simulated as a point mass (mass, m) on the end of a rigid rod (length, r) of zero mass (see Supplemental Experimental Procedures). Visual feedback of the tool (dark gray figure) was provided and updated in real time. Subjects grasped the tool by the circular handle, which was aligned with their hand. The task involved rotating the tool 40° from a starting angle (light gray bar) to a target angle (black bar) while maintaining the handle within a circular home region (light gray). Rotation generated translational forces (F) and rotational torques (τ) at the handle. Figure shows a grayscale version of actual visual feedback presented to subjects (scale bar represents 1 cm). Annotations have been added.

Figure 2

Tool Dynamics Cued by Visual Feedback (A) Direction of peak anticipatory forces as a function of visual orientation of the tool. Data points are circular means (±1 circular standard error [SE]) across subjects (n = 8). Dotted line shows force direction that would fully compensate for the tool dynamics at that orientation (based on simulations; see Results). (B) Peak magnitude of anticipatory forces as a function of visual orientation of the tool. Data points are means (±1 SE) across subjects (n = 8). (C) Peak magnitude of anticipatory forces across experimental blocks. Each data point is the mean of 4 trials averaged across subjects. Shaded region is ±1 SE.

Figure 3

Performance after Training at a Single Orientation and Subsequent Transfer to Novel Orientations (A) Anticipatory forces during probe trials (expressed as compensated object mass) at the training orientation (0°, squares) and transfer orientation (−90°, circles) for tools of different mass. Tool mass is expressed as percentage of each subject's body mass. Compensated object mass is the tool mass (also expressed as percentage body mass) for which the forces would have been appropriate. Data points are means (±1 SE) across subjects (n = 9). Lines show average of individual linear regressions across subjects. (B) Peak handle displacement angle (PDA) during probe trials as a function of visually presented orientation of the tool. Dotted line shows the direction that would fully compensate for the tool dynamics (plotted as in Figure 2A). Data points are circular means (±1 circular SE) across subjects (n = 8) at the training orientation (0°, square) and transfer orientations (−22.5°, −45°, −90°, 180°, circles). (C) Peak handle displacement (independent of direction) during probe trials that shows the transfer of training as a function of the visually presented orientation of the tool. Data points are means (±1 SE) across subjects (n = 8) at the training orientation (0°, square) and transfer orientations (−22.5°, −45°, −90°, 180°, circles). The orientation-dependent decrease in displacement was fit by a half Gaussian for each subject, and the line shows the average fit across subjects (mean fit standard deviation [SD] = 34°). (D) Increases in peak displacement following probe trials that show effects of partial deadaptation as a function of visually presented orientation of the tool, plotted as in (C). Values are means of the subject-by-subject difference between preprobe and postprobe displacements at the training orientation (0°) after partial deadaptation with probe trials at different visually presented orientations of the tool. As in (C), the orientation-dependent decrease in displacement was fit by a half Gaussian for each subject, and the line shows the average fit across subjects (mean fit SD = 39°).

Figure 4

Performance during Exposure when Vision is Either Congruent or Incongruent with Dynamics (A) Peak handle displacement (independent of direction) averaged across subjects (n = 7) for each block (4 trials) for a visually congruent tool at 0° (see inset; a second group experienced a visually congruent tool at 180°). Shaded region is ±1 SE. Black trace shows performance during exposure to full dynamics. Gray traces show pre- and postexposure phases with no translational forces. (B) Peak displacement, plotted as in (A), for an incongruent tool with a visual orientation of 180° (0° for dynamics; see inset; a second group experienced an incongruent tool with vision at 0° and dynamics at 180°). (C) Peak anticipatory force vectors for a congruent tool at 180°. Black arrow shows circular mean (across subjects) for peak forces produced by the dynamics of the tool. Dark and light gray lines show circular mean (across subjects) for peak anticipatory forces at the training (dark gray) and −90° transfer (light gray) orientations. Ellipses show 99% confidence intervals across subjects. (D) Peak anticipatory force vectors for an incongruent tool at 180°, plotted as in (C). (E) PDA during entire pre-exposure phase (48 trials) and for early (first 12 of 48 trials) and late (last 12 of 48 trials) stages of postexposure (deadaptation) phase, for the congruent tools. Dotted line shows PDA predicted from congruent vision and dynamics (V&D) of tool. Data points are circular means (±1 circular SE) across subjects. (F) PDA for the incongruent tools, plotted as in (E). Dotted lines show separate PDA predicted from dynamics (D) and vision (V) of the incongruent tools.