The molecular triggers leading to virulence of a number of human-adapted commensal bacteria such as Streptococcus gallolyticus are largely unknown. This opportunistic pathogen is responsible for endocarditis in the elderly and associated with colorectal cancer. Colonization of damaged host tissues with exposed collagen, such as cardiac valves and pre-cancerous polyps, is mediated by appendages referred to as Pil1 pili. Populations of S. gallolyticus are heterogeneous with the majority of cells weakly piliated while a smaller fraction is hyper piliated. We provide genetic evidences that heterogeneous pil1 expression depends on a phase variation mechanism involving addition/deletion of GCAGA repeats that modifies the length of an upstream leader peptide. Synthesis of longer leader peptides potentiates the transcription of the pil1 genes through ribosome-induced destabilization of a premature stem-loop transcription terminator. This study describes, at the molecular level, a new regulatory mechanism combining phase variation in a leader peptide-encoding gene and transcription attenuation. This simple and robust mechanism controls a stochastic heterogeneous pilus expression, which is important for evading the host immune system while ensuring optimal tissue colonization.