A new general analytical approach for modeling patterns of genetic differentiation and effective size of subdivided populations over time

Abstract

The main purpose of this paper is to develop a theoretical framework for assessing effective population size and genetic divergence in situations with structured populations that consist of various numbers of more or less interconnected subpopulations. We introduce a general infinite allele model for a diploid, monoecious and subdivided population, with subpopulation sizes varying over time, including local subpopulation extinction and recolonization, bottlenecks, cyclic census size changes or exponential growth. Exact matrix analytic formulas are derived for recursions of predicted (expected) gene identities and gene diversities, identity by descent and coalescence probabilities, and standardized variances of allele frequency change. This enables us to compute and put into a general framework a number of different types of genetically effective population sizes (Ne) including variance, inbreeding, nucleotide diversity, and eigenvalue effective size. General expressions for predictions (gST) of the coefficient of gene differentiation GST are also derived. We suggest that in order to adequately describe important properties of a subdivided population with respect to allele frequency change and maintenance of genetic variation over time, single values of gST and Ne are not enough. Rather, the temporal dynamic patterns of these properties are important to consider. We introduce several schemes for weighting subpopulations that enable effective size and expected genetic divergence to be calculated and described as functions of time, globally for the whole population and locally for any group of subpopulations. The traditional concept of effective size is generalized to situations where genetic drift is confounded by external sources, such as immigration and mutation. Finally, we introduce a general methodology for state space reduction, which greatly decreases the computational complexity of the matrix analytic formulas.

Keywords: Coefficient of gene differentiation; Effective population size; Matrix analytic methods; State space reduction; Subpopulation weights.