Nonspecific probe binding and automatic gating in flow cytometry and fluorescence activated cell sorting (FACS)

Math Biosci Eng. 2019 May 21;16(5):4477-4490. doi: 10.3934/mbe.2019223.


Flow cytometry is extensively used in cell biology to differentiate cells of interest (mutants) from control cells (wild-types). For mutant cells characterized by expression of a distinct membrane surface structure, fluorescent marker probes can be designed to bind specifically to these structures while the cells are in suspension, resulting in a sufficiently high fluorescence intensity measurement by the cytometer to identify a mutant cell. However, cell membranes may have relatively weak, nonspecific binding affinity to the probes, resulting in false positive results. Furthermore, the same effect would be present on mutant cells, allowing both specific and nonspecific binding to a single cell. We derive and analyze a kinetic model of fluorescent probe binding dynamics by tracking populations of mutant and wild-type cells with differing numbers of probes bound specifically and nonspecifically. By assuming the suspension is in chemical equilibrium prior to cytometry, we use a two-species Langmuir adsorption model to analyze the confounding effects of non-specific binding on the assay. Furthermore, we analytically derive an expectation maximization method to infer an appropriate estimate of the total number of mutant cells as an alternative to existing, heuristic methods. Lastly, using our model, we propose a new method to infer physical and experimental parameters from existing protocols. Our results provide improved ways to quantitatively analyze flow cytometry data.

Keywords: FACS; Langmuir adsorption; automatic gating; flow cytometry; fluorescing antibodies; mixture model; serial dilution.

Publication types

  • Research Support, N.I.H., Extramural
  • Research Support, U.S. Gov't, Non-P.H.S.

MeSH terms

  • Algorithms
  • Binding Sites
  • Cell Count / statistics & numerical data
  • Cell Membrane / metabolism
  • Cell Separation / methods*
  • Cell Separation / statistics & numerical data
  • Flow Cytometry / methods*
  • Flow Cytometry / statistics & numerical data
  • Fluorescent Dyes / metabolism*
  • Humans
  • Kinetics
  • Mathematical Concepts
  • Models, Biological
  • Mutation


  • Fluorescent Dyes