Individuals in a population often have different fitnesses even when they have identical genotypes, but the effect of this variation on the evolution of a population through complicated fitness landscapes is unknown. Here, we investigate how populations with non-genetic fitness variation cross fitness valleys, common barriers to adaptation in rugged fitness landscapes in which a population must pass through a deleterious intermediate to arrive at a final advantageous stage. We develop a stochastic computational model describing the dynamics of an asexually reproducing population crossing a fitness valley, in which individuals of the same evolutionary stage can have variable fitnesses. We find that fitness variation that persists over multiple generations increases the rate of valley crossing through a novel evolutionary mechanism different from previously characterized mechanisms such as stochastic tunneling. By reducing the strength of selection against deleterious intermediates, persistent fitness variation allows for faster adaptation through rugged fitness landscapes.