Females of many species mate with more than one male (polyandry), yet the adaptive significance of polyandry is poorly understood. One hypothesis to explain the widespread occurrence of multiple mating is that it may allow females to utilize post-copulatory mechanisms to reduce the risk of fertilizing their eggs with sperm from incompatible males. Selfish genetic elements (SGEs) are ubiquitous in eukaryotes, frequent sources of reproductive incompatibilities, and associated with fitness costs. However, their impact on sexual selection is largely unexplored. In this review we examine the link between SGEs, male fertility and sperm competitive ability. We show there is widespread evidence that SGEs are associated with reduced fertility in both animals and plants, and present some recent data showing that males carrying SGEs have reduced paternity in sperm competition. We also discuss possible reasons why male gametes are particularly vulnerable to the selfish actions of SGEs. The widespread reduction in male fertility caused by SGEs implies polyandry may be a successful female strategy to bias paternity against SGE-carrying males.