CAGI5: Objective performance assessments of predictions based on the Evolutionary Action equation

Hum Mutat. 2019 Sep;40(9):1436-1454. doi: 10.1002/humu.23873. Epub 2019 Aug 7.


Many computational approaches estimate the effect of coding variants, but their predictions often disagree with each other. These contradictions confound users and raise questions regarding reliability. Performance assessments can indicate the expected accuracy for each method and highlight advantages and limitations. The Critical Assessment of Genome Interpretation (CAGI) community aims to organize objective and systematic assessments: They challenge predictors on unpublished experimental and clinical data and assign independent assessors to evaluate the submissions. We participated in CAGI experiments as predictors, using the Evolutionary Action (EA) method to estimate the fitness effect of coding mutations. EA is untrained, uses homology information, and relies on a formal equation: The fitness effect equals the functional sensitivity to residue changes multiplied by the magnitude of the substitution. In previous CAGI experiments (between 2011 and 2016), our submissions aimed to predict the protein activity of single mutants. In 2018 (CAGI5), we also submitted predictions regarding clinical associations, folding stability, and matching genomic data with phenotype. For all these diverse challenges, we used EA to predict the fitness effect of variants, adjusted to specifically address each question. Our submissions had consistently good performance, suggesting that EA predicts reliably the effects of genetic variants.

Keywords: deleterious mutation; disease classification; disease driver genes; evolutionary trace; fitness effect; genetic variation; genome interpretation; mutational evolutionary action; single-nucleotide polymorphism (SNP); variants of unknown significance (VUS).

Publication types

  • Research Support, N.I.H., Extramural

MeSH terms

  • Computational Biology / methods*
  • Databases, Genetic
  • Evolution, Molecular
  • Genetic Fitness
  • Genetic Predisposition to Disease
  • Genetic Variation*
  • Humans
  • Phenotype
  • Sequence Alignment
  • Sequence Analysis, DNA / methods*