Many populations of the seed-harvester ant Pogonomyrmex barbatus exhibit genetic caste determination (GCD) generated by the interbreeding of two distinct yet interdependent lineages. Same-lineage matings are genetically predestined to become female reproductives (gynes) whereas alternate-lineage matings become workers. The perpetuation of this system requires that reproductives of both lineages are available for mating and are thus part of the effective population. We label these dependent lineage populations, because each lineage depends on the alternate lineage for worker production. Here we investigate the potential costs associated with GCD in a population with highly skewed lineage frequencies. We reared colonies using newly mated queens from a GCD population and an ecologically equivalent Pogonomyrmex rugosus population with environmental caste determination. GCD founding queens suffer a genetic load from mating randomly and produce fewer brood with advanced development compared with environmental caste determination queens. Our results indicate that GCD queens acquiring a high proportion of same-lineage sperm are unlikely to found a colony successfully. Given model parameters of random mating and founding queens mating with three males on average, there was a close fit between theoretical expectations of variation in colony worker production based on mating and lineage frequencies and empirical deficits in worker production. As expected, severely decreased worker production was specific to the common lineage, suggesting that negative frequency-dependent selection acts to stabilize a dependent lineage system.