GPU based real-time instrument tracking with three-dimensional ultrasound

Abstract

Real-time three-dimensional ultrasound enables new intracardiac surgical procedures, but the distorted appearance of instruments in ultrasound poses a challenge to surgeons. This paper presents a detection technique that identifies the position of the instrument within the ultrasound volume. The algorithm uses a form of the generalized Radon transform to search for long straight objects in the ultrasound image, a feature characteristic of instruments and not found in cardiac tissue. When combined with passive markers placed on the instrument shaft, the full position and orientation of the instrument is found in 3D space. This detection technique is amenable to rapid execution on the current generation of personal computer graphics processor units (GPU). Our GPU implementation detected a surgical instrument in 31 ms, sufficient for real-time tracking at the 25 volumes per second rate of the ultrasound machine. A water tank experiment found instrument orientation errors of 1.1 degrees and tip position errors of less than 1.8mm. Finally, an in vivo study demonstrated successful instrument tracking inside a beating porcine heart.

Figure 1

Schematic (A) and ultrasound image (B) of an atrial septal defect repair. In this procedure a patch is inserted into the heart to cover the atrial septal defect. An anchor driver is also inserted to attach the patch to the septum.

Figure 2

Example of the Radon transform detection of the instrument axis. Each image (A–E) is a projection of the ultrasound image along the corresponding direction shown in the schematic. The projection is along the axis of the instrument (C) is the brightest. Note that this diagram omits out-of-plane projections that are part of the implementation.

Figure 3

Picture (A) and ultrasound image (B) of a minimally invasive anchor driver with passive markers. The instrument tip and roll angle is calculated using the distances x₁ and x_2.

Figure 4

Passive marker template used to identify the location of the three markers on the instrument.

Figure 5

3D Ultrasound image of an instrument with white dots indicating tracked passive markers.

Figure 6

In the tank study instruments were imaged at different φ_Inst. Images were taken of the instrument at five positions for each orientation.

Figure 7

The plot shows the mean angle φ_inst calculated by the tracking algorithm for all 5 position. Errors bar indicate standard deviation, and dashed line shows equality.

Figure 8

Tip position accuracy from tank trials. The distance from the tip position calculated by tracking algorithm to actual tip position is shown for each angle φ_inst.

Figure 9

In vivo x, y, and z position of the instrument tip as reported by an electromagnetic tracker (dashed line) and the tracking algorithm (solid line).

Figure 10

Ultrasound images of the instrument inside a beating porcine heart. The red dots indicate the instrument position calculated by the tracking algorithm. Each image (A–F) are images taken each second for 6 seconds.