Measuring and testing genetic differentiation with ordered versus unordered alleles

Abstract

Estimates and variances of diversity and differentiation measures in subdivided populations are proposed that can be applied to haplotypes (ordered alleles such as DNA sequences, which may contain a record of their own histories). Hence, two measures of differentiation can be compared for a single data set: one (GST) that makes use only of the allelic frequencies and the other (NST) for which similarities between the haplotypes are taken into account in addition. Tests are proposed to compare NST and GST with zero and with each other. The difference between NST and GST can be caused by several factors, including sampling artefacts, unequal effect of mutation rates and phylogeographic structure. The method presented is applied to a published data set where a nuclear DNA sequence had been determined from individuals of a grasshopper distributed in 24 regions of Europe. Additional insights into the genetic subdivision of these populations are obtained by progressively combining related haplotypes and reanalyzing the data each time.