Metal-halide perovskite nanocrystals have demonstrated excellent optoelectronic properties for light-emitting applications. Isovalent doping with various metals (M2+) can be used to tailor and enhance their light emission. Although crucial to maximize performance, an understanding of the universal working mechanism for such doping is still missing. Here, we directly compare the optical properties of nanocrystals containing the most commonly employed dopants, fabricated under identical synthesis conditions. We show for the first time unambiguously, and supported by first-principles calculations and molecular orbital theory, that element-unspecific symmetry-breaking rather than element-specific electronic effects dominate these properties under device-relevant conditions. The impact of most dopants on the perovskite electronic structure is predominantly based on local lattice periodicity breaking and resulting charge carrier localization, leading to enhanced radiative recombination, while dopant-specific hybridization effects play a secondary role. Our results suggest specific guidelines for selecting a dopant to maximize the performance of perovskite emitters in the desired optoelectronic devices.