Evaluation of contactless human-machine interface for robotic surgical training

Int J Comput Assist Radiol Surg. 2018 Jan;13(1):13-24. doi: 10.1007/s11548-017-1666-6. Epub 2017 Sep 15.


Purpose: Teleoperated robotic systems are nowadays routinely used for specific interventions. Benefits of robotic training courses have already been acknowledged by the community since manipulation of such systems requires dedicated training. However, robotic surgical simulators remain expensive and require a dedicated human-machine interface.

Methods: We present a low-cost contactless optical sensor, the Leap Motion, as a novel control device to manipulate the RAVEN-II robot. We compare peg manipulations during a training task with a contact-based device, the electro-mechanical Sigma.7. We perform two complementary analyses to quantitatively assess the performance of each control method: a metric-based comparison and a novel unsupervised spatiotemporal trajectory clustering.

Results: We show that contactless control does not offer as good manipulability as the contact-based. Where part of the metric-based evaluation presents the mechanical control better than the contactless one, the unsupervised spatiotemporal trajectory clustering from the surgical tool motions highlights specific signature inferred by the human-machine interfaces.

Conclusions: Even if the current implementation of contactless control does not overtake manipulation with high-standard mechanical interface, we demonstrate that using the optical sensor complete control of the surgical instruments is feasible. The proposed method allows fine tracking of the trainee's hands in order to execute dexterous laparoscopic training gestures. This work is promising for development of future human-machine interfaces dedicated to robotic surgical training systems.

Keywords: Contactless teleoperation; Hand tracking; Human–machine interface; Robotic surgical training; Unsupervised trajectory analysis.

Publication types

  • Evaluation Study

MeSH terms

  • Gestures
  • Humans
  • Robotic Surgical Procedures / education*
  • Robotic Surgical Procedures / methods
  • User-Computer Interface*