Cell-to-cell variation in gene expression levels (noise) generates phenotypic diversity and is an important phenomenon in evolution, development and disease. TATA-box binding protein (TBP) is an essential factor that is required at virtually every eukaryotic promoter to initiate transcription. While the presence of a TATA-box motif in the promoter has been strongly linked with noise, the molecular mechanism driving this relationship is less well understood. Through an integrated analysis of multiple large-scale data sets, computer simulation and experimental validation in yeast, we provide molecular insights into how noise arises as an emergent property of variable binding affinity of TBP for different promoter sequences, competition between interaction partners to bind the same surface on TBP (to either promote or disrupt transcription initiation) and variable residence times of TBP complexes at a promoter. These determinants may be fine-tuned under different conditions and during evolution to modulate eukaryotic gene expression noise.