Enabling User-Guided Segmentation and Tracking of Surface-Labeled Cells in Time-Lapse Image Sets of Living Tissues
Item in Clipboard
Enabling User-Guided Segmentation and Tracking of Surface-Labeled Cells in Time-Lapse Image Sets of Living Tissues
To study the process of morphogenesis, one often needs to collect and segment time-lapse images of living tissues to accurately track changing cellular morphology. This task typically involves segmenting and tracking tens to hundreds of individual cells over hundreds of image frames, a scale that would certainly benefit from automated routines; however, any automated routine would need to reliably handle a large number of sporadic, and yet typical problems (e.g., illumination inconsistency, photobleaching, rapid cell motions, and drift of focus or of cells moving through the imaging plane). Here, we present a segmentation and cell tracking approach based on the premise that users know their data best-interpreting and using image features that are not accounted for in any a priori algorithm design. We have developed a program, SeedWater Segmenter, that combines a parameter-less and fast automated watershed algorithm with a suite of manual intervention tools that enables users with little to no specialized knowledge of image processing to efficiently segment images with near-perfect accuracy based on simple user interactions.
Copyright © 2012 International Society for Advancement of Cytometry.
Common segmentation difficulties in confocal images of living, cadherin-GFP stained fruit fly embryos. The large cells on the right of the image are amnioserosa cells; the smaller ones at the left are epidermal.
(A) is an unsegmented image and (B) is the same image with an overlay of seeds ( small green squares) generated automatically by application of a Gaussian filter (σ = 2.5 μm) and segment outlines ( red lines) generated by a watershed algorithm. The numbered arrows point to several common errors in automatic segmentation. (1) An object obscures the view of the cell edge. (2) A single cell is divided between two seeds, i.e., oversegmentation. (3) Two cells share a single seed, i.e., undersegmentation. (4) A region that should be part of the image background instead receives seeds and is assigned as cells. (5) An area of epidermis cells that is very badly mis-segm ented because the Gaussian filter is too large for these smaller cells. The user must decide if segmentation of this region should be completely reworked manually or skipped altogether. A smaller Gaussian filter (σ = 0.625 instead of 2.5) will effectively generate seeds for these smaller cells, but at the expense of severely oversegmenting the amnioserosa cells (into ~10 segments each, not shown). (6) Sub-cellular regions are misassigned. One can often determine which cells these regions belong to based on other images in the time-lapse set.
Adding and deleting seeds manually.
(A) Initial automatic segmentation of an image (σ = 2.5 μm). Over-segmented regions with unwanted seeds are circled. The upper circled region highlights a cell at the edge of the imaging plane with a poorly defined boundary. The lower-left circled region has two seeds dividing a single cell. (B) Segmentation after manual removal of unwanted seeds. (C) Segmentation after manual addition of seeds to correct under-segmented regions (cyan fill). Seeds were added for sixteen cells around the margins of the tissue. These cells had been considered part of the background by the automatic algorithm. Seeds were also added for three internal cells that had not automatically received their own seeds.
Adding multiple extra seeds to correct mis-segmentation of cellular subregions.
(A) Original image to be segmented. (B, C) Initial segmentation shown as an overlay with green seeds and red segment boundaries in (B) and as a false-colored cell ID map in (C). The subregion just to the right of the central seed is misassigned to an adjacent cell (blue). The upper left boundary of the central cell (pink) is also not satisfactory. (D, E) By adding a single extra seed, the originally mis-segmented subregion is reassigned to the appropriate cell (pink instead of blue). For the user, this is a two-click process: a left-click on the region that needs to be expanded followed by a right click to place the extra seed. The inset to the left of (D) shows a close-up of a remaining problematic boundary (with and without overlay). (F, G) This boundary is improved by adding a polyline of extra seeds (green line in (F) that appears as distinct seeds in the inset). For the user, creating a line of seeds works as above except multiple right clicks are used. A line segment of seeds is added between each successive right-click location. As shown in the inset to the left of (F), the result is a slight improvement in the overlap of the watershed boundary and the imaged cell-cell boundary. The best location for this boundary was determined by visually tracking its motion in previous and subsequent frames of the image stack (not shown).
Cell tracking when the frame-to-frame movements of cells are large.
(A–C) Complete manually assisted segmentation of a cluster of amnioserosa cells. The segmentation overlay shows seeds ( green squares) and segment outlines ( red lines). (B) is a close-up of the boxed region and (C) is the corresponding false-colored cell ID map. (D–F) Automatic tracking and segmentation of the next frame after laser ablation of a central cell and with a large time interval between frames (70 seconds). The large time interval exaggerates cell motion between frames and causes the centroid-based algorithm to track cells improperly in some regions, especially near the bottom middle of the image (zoomed region in E, F). The errors are clearly discernable in (F) compared to (C). Note that even in this relatively extreme case, the automatic tracking performs very well for most cells in the image, particularly outside the boxed region. Tracking generally works well unless the cell moves over half its diameter. (G–I) Corrected tracking and segmentation after using the “Lasso” and “Move Seeds” tools. The “Lasso” tool works by clicking to form a polygon that encircles multiple seeds. These seeds are then moved using the arrow keys to position them properly. This seed adjustment process is quite fast (a few seconds) by starting with bulk motion and then adjusting individual seeds as needed.
Comparison of segmentation speed and accuracy for a typical data set: 190 frames with an average of 64 cells per frame.
(A, B) Improved accuracy versus time spent on manual intervention using SWS. Both graphs represent the same data; (B) simply has a tighter zoom in the y-axis to more clearly show the data after 50 minutes. Intermediate segmentations were saved after each change in watershed borders or at least every 60 s. The accuracy at each intermediate time point was assessed based on either the percentage of pixels whose assignment matched the final segmentation or the percentage of cells whose boundaries matched the final segmentation. The “first pass” segmentation was performed with minimal tracking, generally letting the algorithms do the work automatically (50 min), achieving a pixel-based accuracy of 98.6% and a cell-based accuracy of 88%. We then performed three more rounds of manual intervention and adjustment that improved the segmentation and tracking to user-desired accuracy in approximately six hours (~ 2 min per frame). The efficacy of manual intervention will vary with user experience and imaging quality, but this set is representative. The diminishing returns of continued manual intervention are most evident in the pixel-based comparison, but even this is somewhat linear over large time periods because we made successive passes through the entire stack. (C) Distribution of errors over all segmented cells for selected intermediate segmentations and other techniques. Errors are defined as deviations from the final SWS segmentation. The x-axis is a normalized list of cell indices sorted from largest to smallest relative error. The y-axis is the number of erroneous pixels for each cell divided by the average area of all cells. We compared a “Hands Off” SWS segmentation with no user intervention and a “First Pass” SWS segmentation with minimal manual tracking assistance. For comparison, we include a 3D watershed segmentation, and SWS on an CLAHE-filtered image set. (D) Distribution of errors over all segmented cells in comparison to a “gold standard” manual segmentation using a vector editor. These comparisons are limited to five evenly spaced frames (every 47 th of the full data set). Absolute errors are thus compared for SWS segmentation, Packing Analyzer, and 3D watershed segmentation. As a baseline, we include errors induced by pixelating the vector segmentation.
Time-and-space pair correlation function of triple-junction velocities for a data set with 66 segmented cells. The
x-axis is the time separation between points (τ) and the y-axis is the distance between points (Δ). Correlations are normalized so that the peak at (0,0) has a value of 1. The Δ=0 axis (autocorrelation function) is plotted above the density plot, and the τ=0 axis (distance correlation) is plotted to the left. Dashed lines in these two plots represent zero correlation. The minima and maxima of the autocorrelation appear as dark and bright spots on the Δ=0 axis of the density plot with the first of each occurring at 123 and 267 s, respectively. In the full pair correlation function (density plot), the extrema move to longer time delays as the distance between the pair increases. This wave-like propagation is demarcated by the angled dashed line, which has a slope and thus velocity of 0.14 μm/s.
3-D Active Meshes: Fast Discrete Deformable Models for Cell Tracking in 3-D Time-Lapse Microscopy
A Dufour et al.
IEEE Trans Image Process 20 (7), 1925-37.
Variational deformable models have proven over the past decades a high efficiency for segmentation and tracking in 2-D sequences. Yet, their application to 3-D time-lapse …
A Robust Algorithm for Segmenting and Tracking Clustered Cells in Time-Lapse Fluorescent Microscopy
W Tarnawski et al.
IEEE J Biomed Health Inform 17 (4), 862-9.
We present herein a robust algorithm for cell tracking in a sequence of time-lapse 2-D fluorescent microscopy images. Tracking is performed automatically via a multiphase …
Automated Profiling of Individual Cell-Cell Interactions From High-Throughput Time-Lapse Imaging Microscopy in Nanowell Grids (TIMING)
A Merouane et al.
Bioinformatics 31 (19), 3189-97.
Fluorescently labeled human T cells, natural killer cells (NK), and various target cells (NALM6, K562, EL4) were co-incubated on polydimethylsiloxane arrays of sub-nanoli …
Untangling Cell Tracks: Quantifying Cell Migration by Time Lapse Image Data Analysis
CM Svensson et al.
Cytometry A 93 (3), 357-370.
Automated microscopy has given researchers access to great amounts of live cell imaging data from in vitro and in vivo experiments. Much focus has been put on extracting …
Segmenting and Tracking Multiple Dividing Targets Using Ilastik
C Haubold et al.
Adv Anat Embryol Cell Biol 219, 199-229.
Tracking crowded cells or other targets in biology is often a challenging task due to poor signal-to-noise ratio, mutual occlusion, large displacements, little discernibi …
PubMed Central articles
Unjamming and Collective Migration in MCF10A Breast Cancer Cell Lines
JH Kim et al.
Biochem Biophys Res Commun 521 (3), 706-715.
Each cell comprising an intact, healthy, confluent epithelial layer ordinarily remains sedentary, firmly adherent to and caged by its neighbors, and thus defines an eleme …
Using a Continuum Model to Decipher the Mechanics of Embryonic Tissue Spreading From Time-Lapse Image Sequences: An Approximate Bayesian Computation Approach
TL Stepien et al.
PLoS One 14 (6), e0218021.
Advanced imaging techniques generate large datasets capable of describing the structure and kinematics of tissue spreading in embryonic development, wound healing, and th …
Elongated Cells Drive Morphogenesis in a Surface-Wrapped Finite-Element Model of Germband Retraction
WT McCleery et al.
Biophys J 117 (1), 157-169.
During Drosophila embryogenesis, the germband first extends to curl around the posterior end of the embryo and then retracts back; however, retraction is not simply the r …
Spinal Neural Tube Closure Depends on Regulation of Surface Ectoderm Identity and Biomechanics by Grhl2
E Nikolopoulou et al.
Nat Commun 10 (1), 2487.
Lack or excess expression of the surface ectoderm-expressed transcription factor Grainyhead-like2 (Grhl2), each prevent spinal neural tube closure. Here we investigate th …
Inferring Statistical Properties of 3D Cell Geometry From 2D Slices
TA Sharp et al.
PLoS One 14 (2), e0209892.
Although cell shape can reflect the mechanical and biochemical properties of the cell and its environment, quantification of 3D cell shapes within 3D tissues remains diff …
Research Support, N.I.H., Extramural
Research Support, Non-U.S. Gov't
Research Support, U.S. Gov't, Non-P.H.S.
Cell Tracking / methods
Image Interpretation, Computer-Assisted
Image Processing, Computer-Assisted
Imaging, Three-Dimensional / methods
Pattern Recognition, Automated
Time-Lapse Imaging / methods
LinkOut - more resources
Full Text Sources Molecular Biology Databases