Loss of hearing or vision has been identified as a significant risk factor for dementia but underlying molecular mechanisms are unknown. In different Drosophila models of blindness, we observe non-autonomous induction of stress granules in the brain and their reversal upon restoration of vision. Stress granules include cytosolic condensates of p62, ATF4 and XRP1. This cytosolic restraint of the ATF4 and XRP1 transcription factors dampens expression of their downstream targets during cellular stress. Cytosolic condensates of p62 and ATF4 were also evident in the thalamus and hippocampus of mouse models of congenital or degenerative blindness. These data indicate conservation of the link between loss of sensory input and dysregulation of stress responses critical for protein quality control in the brain.