Efficient multiple object tracking using mutually repulsive active membranes

Abstract

Studies of social and group behavior in interacting organisms require high-throughput analysis of the motion of a large number of individual subjects. Computer vision techniques offer solutions to specific tracking problems, and allow automated and efficient tracking with minimal human intervention. In this work, we adopt the open active contour model to track the trajectories of moving objects at high density. We add repulsive interactions between open contours to the original model, treat the trajectories as an extrusion in the temporal dimension, and show applications to two tracking problems. The walking behavior of Drosophila is studied at different population density and gender composition. We demonstrate that individual male flies have distinct walking signatures, and that the social interaction between flies in a mixed gender arena is gender specific. We also apply our model to studies of trajectories of gliding Myxococcus xanthus bacteria at high density. We examine the individual gliding behavioral statistics in terms of the gliding speed distribution. Using these two examples at very distinctive spatial scales, we illustrate the use of our algorithm on tracking both short rigid bodies (Drosophila) and long flexible objects (Myxococcus xanthus). Our repulsive active membrane model reaches error rates better than 5 x 10(-6) per fly per second for Drosophila tracking and comparable results for Myxococcus xanthus.

Figure 1. Schematic illustration of the principle of repelling active contours.

(a) When objects are far away and the attractive image potential fields (blue solid lines) don't interact, two active objects (red and cyan circles) correctly fall into the image energy minima. (b) When objects are close, the potential fields (blue dashed lines) overlap and cause dislocated or merged minima. Two active objects converge into the same minimum. (c) With the repulsive potential added (red solid line), the total field the other object is in is recovered (cyan solid line). (d) The same principle can be applied to two and higher dimensions as shown. The black solid ovals represent two contacting objects. The images are blurred to give smooth attraction potential, and with repulsion, two minima can be resolved (green and red).

Figure 2. The trajectories and velocities exacted from the tracking results of five male flies.

(a) A sample image of five flies in the circular arena. Three seconds of walking trajectories of the flies are labeled in different colors. The control points are labeled as the yellow squares. (b) The parallel component of the walking velocity of one fly is plot as a function of time. When the fly jumps (indicated by arrows), the tracker may reverse the orientation and negate the velocity. The reversal can be detected off-line using a Hidden Markov Model (HMM). (c–g) Velocity histograms of five individual flies. The color indicates the the probability density distribution plotted on a logarithmic scale.

Figure 3. Tracking results of in-contact and jumping Drosophila as exceptional conditions, and the histograms of the relative positions between flies.

The positional information of flies from the movies are shown in the three-dimensional kymograph, where the boundary of the fly images are drawn in false color depending on time. The positions of the flies are indicated as short open contours in the same color and connected by lines as visual aid. (a) Two flies moved in proximity to each other and then moved apart, causing the mask and contours to merge and split again. (b) A missing fly image was caused by the a jumping event, followed by re-orientation of the head direction. (c)-(h) The distribution of the relative position between two flies is shown as the logarithm of the distribution probability density. The displacement is measured relative to the first fly on the perpendicular (x) and the parallel direction relative to the head direction (y). (c) Male 1 (M1) relative to female (F), (d) male 2 (M2) relative to female (F), (e) F relative to M1, (f) M2 relative to M1, (g) F relative to M2, and (h) M1 relative to M2.

Figure 4. Bright field images of Myxococcus xanthus cells are taken, and transformed into the probability map that is then used to generate the image potential to interact with the contours.

(a) A field of view of 41 m 41 m (512512 pixels) contains over 200 Myxococcus xanthus cells. The relaxed position of the contours are overlaid on top of the bright field image with false color labeling. (b) Zoomed-in image of a portion in (a). (c,d) The large and the small eigenvalues of the Hessian matrix of (b). (e) The locally averaged eigenvectors indicate the alignment magnitude of features. The sign is chosen such that higher value indicates less order, thus high chance to be the background and vise versa. (f) The distribution of pixels in the classifier coordinate: two principle components from the eigenvalue-intensity space (horizontal axes), and the alignment magnitude (vertical axis). Pixels are categorized into three groups along the three axes, and color-coded in red (ridge), green (valley) and blue (background). The projections along three axis are shown as guides for viewing. (g) After classification, each pixel in the image is color coded in the same way as in (f) according to the probability of being ridge, valley or background. (h) The enhanced image for repulsive active contour model is calculated from the classification probability map shown in (g). (i) Image intensity profile on a line segment (red dashed line in (a)) illustrates the nonuniform contrast at the edge and at the inside of a cell cluster.

Figure 5. A contour can ride across a ridge and cause a marginally stable configuration. Allowing tips to grow solves this problem.

(a) The total growth of the two ends of a contour is determined by the current length and the normal length of the contour. The growth is distributed unevenly to two termini according to the tangential resistance force. The dashed line indicates a hypothetical contour and the two ends are indicated by the round dots. (b) Without the ends shrinking and regrowing, the contours are trapped in a marginally stable configuration, where the contour on the right (indicated by the red dotted line) leaks to the left potential well and squeezed left contour short due to repulsion. (c–f) The length of the contours are shortened initially and let grow to lead to the correct contour positions. The white arrow indicates the direction of the tip growth, and the white dashed lines are the visual guide to help illustrate the growth.

Figure 6. Tracking results and speed statistics of Myxococcus xanthus cells.

(a) Trajectories of 205 cells over 2000 seconds are indicated in different colors, overlaid on the first frame of the movie. (b) The tangential gliding speed along a selected cell is plot against time. Raw speed trace (light red) is smoothed by Gaussian kernel with seconds (dark red). Six directional reversals are identified by the zero-crossings of the smoothed speed curve (pointed by arrows). (c) The histogram of gliding speed magnitude of 205 cells approximately follows an exponential distribution with the mean value at 0.49 m/min.