Continuous Shape Estimation of Continuum Robots Using X-ray Images

Abstract

We present a new method for continuously and accurately estimating the shape of a continuum robot during a medical procedure using a small number of X-ray projection images (e.g., radiographs or fluoroscopy images). Continuum robots have curvilinear structure, enabling them to maneuver through constrained spaces by bending around obstacles. Accurately estimating the robot's shape continuously over time is crucial for the success of procedures that require avoidance of anatomical obstacles and sensitive tissues. Online shape estimation of a continuum robot is complicated by uncertainty in its kinematic model, movement of the robot during the procedure, noise in X-ray images, and the clinical need to minimize the number of X-ray images acquired. Our new method integrates kinematics models of the robot with data extracted from an optimally selected set of X-ray projection images. Our method represents the shape of the continuum robot over time as a deformable surface which can be described as a linear combination of time and space basis functions. We take advantage of probabilistic priors and numeric optimization to select optimal camera configurations, thus minimizing the expected shape estimation error. We evaluate our method using simulated concentric tube robot procedures and demonstrate that obtaining between 3 and 10 images from viewpoints selected by our method enables online shape estimation with errors significantly lower than using the kinematic model alone or using randomly spaced viewpoints.

Fig. 1

Our objective is to accurately estimate the shape of a continuum robot over time during a procedure using a small number of optimally selected 2D X-ray projection images. We assume the X-ray sensor is mounted on a C-arm (top), a commonly used medical device that rotates the X-ray sensor about the patient lying on an operating table. Our method computes optimal viewpoints for the X-ray sensor to maximize the quality of the online shape estimation of the continuum robot. We apply our method to concentric tube robots, a type of continuum robot (bottom).

Fig. 2

Shape of the curvilinear device over time can be represented as a surface. The horizontal axis denotes time and the vertical axis denotes shape of the device in space.

Fig. 3

Scenarios 1 and 2 involve maneuvering a concentric tube robot, which is deployed via a bronchoscope (cyan), through bronchial tubes in a human lung to clinical targets in simulation. We show the actual concentric tube robot shape (green), the kinematic model (blue), and the 3D reconstructed shape using our optimal method (red dots) after each image acquisition and at the final time T. Inlayed are the simulated camera views from the viewpoint selected by our approach. The camera views include the actual concentric tube robot shape (green), simulated noisy segmented points along the image of the actual concentric tube robot that are used for the reconstruction (black dots), and the kinematic model for reference purposes (blue).

Fig. 4

Basis functions for the y-coordinate of the continuum robot.

Fig. 5

Mean error E^M(t) as a function of time for scenarios 1 and 2 for three shape estimation approaches: the kinematic model, our estimation approach with random sampling of viewpoints, and our estimation approach with optimal viewpoint selection. We set N_I = 3 and acquire images at the times of the red bars. The shaded envelope shows the range of the middle 50% of the data.

Fig. 6

Mean average error Ē^M as a function of the number N_I of images acquired for scenarios 1 and 2 for three shape estimation approaches: the kinematic model, our estimation approach with random sampling of viewpoints, and our estimation approach with optimal viewpoint selection. With only a small number of X-ray images, our method can accurately estimate continuum robot shape.