Numerous studies in both prokaryotes and eukaryotes have shown that, under standard growth conditions, less than 20% of the protein-coding genes are essential for survival. This suggests that biological systems have evolved to have a high degree of robustness to mutational disruptions that can affect the majority of their genes. This mutational robustness could arise either due to redundancy, i.e. direct backup, or due to distributed architecture, i.e. indirect backup where multiple genes contribute to the functioning of a process in the system. Despite clear evidence for direct backup, the prevalence of indirect backup is poorly understood. In this study, we reveal the existence of a hidden distributed architecture behind the scale-free transcriptional regulatory network of yeast by applying a unique network transformation procedure and show that the network is tolerant even to mutations that disrupt regulatory hubs. Contrary to what is generally accepted, our observation that hubs can be lost or replaced in evolution suggests that this hidden distributed architecture behind scale-free networks protects the overall transcriptional program of the organism from mutations affecting major regulatory hubs. We show that the distributed architecture has been provided by an unexpectedly large number of coordinating partners for any regulatory protein. On the basis of these findings, we propose that the existence of such architecture can allow organisms to explore the adaptive landscape in changing environments by providing the plasticity required to reprogram levels of expression of specific genes that may enhance survival. Thus, an "over-engineered" backup system in the form of distributed architecture is likely to be a major determinant of the "evolvability" of the gene expression in organisms faced with environmental diversity.