Bacteria with elevated mutation rates are frequently found among natural isolates. This is probably because of their ability to generate genetic variability, the substrate for natural selection. However, such high mutation rates can lead to the loss of vital functions. The evolution of bacterial populations may happen through alternating periods of high and low mutation rates. The cost and benefits of high mutation rates in the course of bacterial adaptive evolution are reviewed.