Mathematical properties of Fst between admixed populations and their parental source populations

Abstract

We consider the properties of the F(st) measure of genetic divergence between an admixed population and its parental source populations. Among all possible populations admixed among an arbitrary set of parental populations, we show that the value of F(st) between an admixed population and a specific source population is maximized when the admixed population is simply the most distant of the other source populations. For the case with only two parental populations, as a function of the admixture fraction, we further demonstrate that this F(st) value is monotonic and convex, so that F(st) is informative about the admixture fraction. We illustrate our results using example human population-genetic data, showing how they provide a framework in which to interpret the features of F(st) in admixed populations.