The role of genetic instability in driving carcinogenesis remains controversial. Genetic instability should accelerate carcinogenesis by increasing the rate of advantageous driver mutations; however, genetic instability can also potentially retard tumour growth by increasing the rate of deleterious mutation. As such, it is unclear whether genetically unstable clones would tend to be more selectively advantageous than their genetically stable counterparts within a growing tumour. Here, we show the circumstances where genetic instability evolves during tumour progression towards cancer. We employ a Wright-Fisher type model that describes the evolution of tumour subclones. Clones can acquire both advantageous and deleterious mutations, and mutator mutations that increase a cell's intrinsic mutation rate. Within the model, cancers evolve with a mutator phenotype when driver mutations bestow only moderate increases in fitness: very strong or weak selection for driver mutations suppresses the evolution of a mutator phenotype. Genetic instability occurs secondarily to selectively advantageous driver mutations. Deleterious mutations have relatively little effect on the evolution of genetic instability unless selection for additional driver mutations is very weak or if deleterious mutations are very common. Our model provides a framework for studying the evolution of genetic instability in tumour progression. Our analysis highlights the central role of selection in shaping patterns of mutation in carcinogenesis.
Keywords: carcinogenesis; clonal expansion; genetic instability; mathematical biology; selection.