A Markerless 3D Computerized Motion Capture System Incorporating a Skeleton Model for Monkeys

Abstract

In this study, we propose a novel markerless motion capture system (MCS) for monkeys, in which 3D surface images of monkeys were reconstructed by integrating data from four depth cameras, and a skeleton model of the monkey was fitted onto 3D images of monkeys in each frame of the video. To validate the MCS, first, estimated 3D positions of body parts were compared between the 3D MCS-assisted estimation and manual estimation based on visual inspection when a monkey performed a shuttling behavior in which it had to avoid obstacles in various positions. The mean estimation error of the positions of body parts (3-14 cm) and of head rotation (35-43°) between the 3D MCS-assisted and manual estimation were comparable to the errors between two different experimenters performing manual estimation. Furthermore, the MCS could identify specific monkey actions, and there was no false positive nor false negative detection of actions compared with those in manual estimation. Second, to check the reproducibility of MCS-assisted estimation, the same analyses of the above experiments were repeated by a different user. The estimation errors of positions of most body parts between the two experimenters were significantly smaller in the MCS-assisted estimation than in the manual estimation. Third, effects of methamphetamine (MAP) administration on the spontaneous behaviors of four monkeys were analyzed using the MCS. MAP significantly increased head movements, tended to decrease locomotion speed, and had no significant effect on total path length. The results were comparable to previous human clinical data. Furthermore, estimated data following MAP injection (total path length, walking speed, and speed of head rotation) correlated significantly between the two experimenters in the MCS-assisted estimation (r = 0.863 to 0.999). The results suggest that the presented MCS in monkeys is useful in investigating neural mechanisms underlying various psychiatric disorders and developing pharmacological interventions.

Fig 1. Markerless MCS for monkeys.

A: Experimental setup consisting of a monkey cage with four depth cameras. B: Schematic illustration of processing steps of the present MCS. A monkey was captured by four depth cameras (Cam1-4) (a, b), and the images were merged to make a 3D image of the monkey represented by 3D points on the entire surface of the monkey (b). Simultaneously captured color images were mapped onto the 3D points (c). Finally, a skeleton model of the monkey was fitted onto the 3D image (d). C: A skeletal model of a monkey used in the present study. The model consisted of spheres connected by joints. Centers of the spheres, where lines are connected, indicates joints. Number of degrees of freedom (DOF) in each joint is shown by color. D: Attraction force from the points. Small squares represent captured 3D points. Gray spheres represent spheres in the model. The red points attract the sphere i. E: Repulsive force from the points. Arrows indicate the surface normal at the points. The blue points push the sphere i away. Other descriptions are same as D.

Fig 2. Examples of captured motion in the shuttling task.

A: Snapshots of the video captured in the task without obstacles (session 1), the task with obstructing bars at a low height in the middle of the cage (session 2), and the task with obstructing bars at a medium height in the middle of the cage (session 3). White solid lines indicate inner skeletons in the trunk and right limbs, dotted lines indicate inner skeletons in the left limbs. B and C: Traces of the estimated posture from the side view (B) and top view (C) based on the snapshots shown in A. Black bars and points represent the obstacle bars. Green lines, trunk; black lines, head; red lines, forelimbs; blue lines, hind limbs. The solid and dotted lines represent right and left limbs, respectively.

Fig 3. Validation of behavioral event detection by the MCS.

A: Chronograms of behavioral events in the MCS-assisted estimation (MCS, blue) and manual estimation based on visual inspection (Exp, red) in session 2 (left) and session 3 (right). Note that there was no false positive nor false negative detection in the chronograms. The monkey displayed crawling once, but did not cross the bars in session 2. In the 6th trial in session 3 (arrow), the monkey crossed the bars without crawling, i.e., it passed between the bars. B: Correlation of the duration of behavioral events between MCS-assisted and manual estimation. Values in each graph indicate the correlation coefficient (r) and p-value of the correlation (p).

Fig 4. Comparison of behavioral event detection errors between the two different experimenters in MCS-assisted and manual estimation in the shuttling task.

Estimation errors of onset and offset timings and duration in the shuttling task were compared. * Significant difference, p < 0.05 (paired t-test). Error bars represent SEMs.

Fig 5. Effects of MAP on spontaneous behaviors.

A and B: Examples of the time course of head rotation speed (black line) and chest speed (gray line) of a monkey after administration of saline (A) and MAP (B). Thick black bars above the graph represent periods when the monkey was crouching. C-E: Comparison of motor activities between saline and MAP in the MCS-assisted estimation. * Significant difference, p < 0.05. + Tended toward significance, p < 0.1.

Fig 6. Reproducibility of estimated data using the MCS in the MAP experiment.

A-C: Correlation of motor activities in the MAP experiment between the two different experimenters in MCS-assisted estimation.