Bidirectional local distance measure for comparing segmentations

Abstract

Purpose: To accurately quantify the local difference between two contour surfaces in two- or three-dimensional space, a new, robust point-to-surface distance measure is developed.

Methods: To evaluate and visualize the local surface differences, point-to-surface distance measures have been utilized. However, previously well-known point-to-surface distance measures have critical shortfalls. Previous distance measures termed "normal distance (ND)," "radial distance," or "minimum distance (MD)" can report erroneous results at certain points where the surfaces under comparison meet certain conditions. These skewed results are due to the monodirectional characteristics of these methods. ComGrad distance was also proposed to overcome asymmetric characteristics of previous point-to-surface distance measures, but their critical incapability of dealing with a fold or concave contours. In this regard, a new distance measure termed the bidirectional local distance (BLD) is proposed which minimizes errors of the previous methods by taking into account the bidirectional characteristics with the forward and backward directions. BLD measure works through three steps which calculate the maximum value between the forward minimum distance (FMinD) and the backward maximum distance (BMaxD) at each point. The first step calculates the FMinD as the minimum distance to the test surface from a point, p(ref) on the reference surface. The second step involves calculating the minimum distances at every point on the test surface to the reference surface. During the last step, the BMaxD is calculated as the maximum distance among the minimum distances found at p(ref) on the reference surface. Tests are performed on two- and three-dimensional artificial contour sets in comparison to MD and ND measure techniques. Three-dimensional tests performed on actual liver and head-and-neck cancer patients.

Results: The proposed BLD measure provides local distances between segmentations, even in situations where ND, MD, or ComGrad measures fail. In particular, the standard deviation measure is not distorted at certain geometries where ND, MD, and ComGrad measures report skewed results.

Conclusions: The proposed measure provides more reliable statistics on contour comparisons. From the statistics, specific local and global distances can be extracted. Bidirectional local distance is a reliable distance measure in comparing two- or three-dimensional organ segmentations.

FIG. 1.

Shortfalls of normal and minimum distance measures: (a) ND cannot be found at the point p₁ since the normal vector on the reference surface does not intersect with the test surface. The ND measure may report longer distances when two surfaces are not parallel such as at the point p₂. (b) The MD measure may report shorter distances when the test surface is winding and the reference surface is flat as highlighted circle. Although the local discrepancy between the surfaces in the circle area is larger at B₁ than B₂, the MD at the points, p₃ and p₄ will be reported as the same as at points p_t1 and p_t2, respectively. The reference surface R is drawn as a solid line and the test surface T is drawn as dotted line.

FIG. 2.

An example illustrates the BLD measure scheme; FMinD at the point p_ref is marked as ①. During the BMaxD calculation ② and ③ are the candidates. Since BMaxD selects the maximum distance among the candidates, ② is chosen as BMaxD. The final BLD(p_ref, T) is the maximum between FMinD and BMaxD. Therefore, BMaxD, ② is chosen as the final BLD(p_ref, T) in this example.

FIG. 3.

Analyses of measured distances between shifted contours along the contour perimeter position. (a)–(d) Distances measured for BLD, ND, ComGrad, and MD. (c) Distances profile at each pixel on the reference contour. Solid lines represent the BLD measure, and the dotted lines represent the MD measure at every point on the reference contour. The dashed lines show the ND measure. The long dashed orange lines show the ComGrad measure. The ND erroneously measures longer than 20 mm shifts at several points (shown in three regions as marked with A, B, E). MD erroneously measures shorter than 20 mm at two regions as marked with C and D. The ComGrad measures longer than 20 mm shifts at the region as marked with E.

FIG. 4.

Comparison of ND, MD, ComGrad, and BLD measures: Case 1—high similarity and weakly winding contours; Case 2—low similarity and weakly winding contours; Case 3—medium similarity and strongly winding contours; Case 4—winding contours with concave and folded curves. The solid lines are reference contours and the dashed lines are test contours. For easy illustration, the linked lines between contours indicate measured distances.

FIG. 5.

Two liver contours from two treatment plans are constructed as mesh structures. After rigid-body image registration and smoothing are performed, BLD and ND distances are displayed in a color map. (a) Two 3D livers are reconstructed from the first and second plans. (b) BLD measure and (c) ND measure. BLD measure reports distances in these areas up to 29 mm. However, ND measure calculates the distances in these areas up to 117 mm. [URL: http://dx.doi.org/10.1118/1.4754802.1]

FIG. 6.

Distances between two clinical target volumes from two head-and-neck treatment plans are measured. Distances between contours are displayed in a color map after rigid-body image registration and smoothing. (a) 3D CTV structures from the head-and-neck case. Two 3D head-and-neck cases are reconstructed from the first and second plans. (b) BLD measure and (c) ND measure. The maximum distance measured by BLD measure is 17 mm, but ND measure reported the maximum distance of 32 mm.

FIG. 7.

The effect of Gaussian smoothing is compared between our proposed BLD measure and the ND measure.

FIG. 8.

Distances between two contours displayed as histograms. Three different distance measures are applied to the same contours. (a) Distance histogram comparison without smoothing—the MD measure showed high frequency at short distances and the ND measure shows erroneous measurements beyond 30 mm (compressed at 30 mm for readability) not observed in other histograms calculated using other measures. (b)–(d) Analyses are performed following three different levels of volume surface smoothing.

FIG. 9.

A standard deviation map built from nine expert's contours for a prostate. (a) Nine expert's contours. (b) Standard deviation map.