Is self-fertilization an evolutionary dead end?

Abstract

A compound hypothesis positing that self-fertilization is an evolutionary dead end conflates two distinct claims: the transition from outcrossing to selfing is unidirectional; and the diversification rate, or the balance of the speciation and extinction rate, is negative for selfing species. Both claims have enjoyed widespread informal support for decades, but have recently come under suspicion. Sources of data that apparently contradict strongly asymmetric mating system transitions often rely on statistical phylogenetic tests plagued by profound flaws. Although recently developed models mend preceding approaches, they have been employed sparingly, and many problems remain. Theoretical investigations, genetic data and applications of new phylogenetic methods provide indirect support for an association of selfing with negative diversification rates. We lack direct tests of reversals from selfing to outcrossing, and require data concerning the genetic basis and complexity of independently evolved outcrossing adaptations. The identification of the mechanisms that limit the longevity of selfing lineages has been difficult. Limitations may include brief and variable durations of selfing lineages, as well as ongoing difficulties in relating additive genetic and nucleotide variation. Furthermore, a common line of evidence for the stability of mixed mating - based simply on its frequent occurrence - is misleading. We make specific suggestions for research programs that aim to provide a richer understanding of mating system evolution and seriously challenge Stebbins' venerable hypothesis.