Stress granules (SGs) are formed in the cytosol as an acute response to environmental cues and activation of the integrated stress response (ISR), a central signaling pathway controlling protein synthesis. Using chronic virus infection as stress model, we previously uncovered a unique temporal control of the ISR resulting in recurrent phases of SG assembly and disassembly. Here, we elucidate the molecular network generating this fluctuating stress response by integrating quantitative experiments with mathematical modeling and find that the ISR operates as a stochastic switch. Key elements controlling this switch are the cooperative activation of the stress-sensing kinase PKR, the ultrasensitive response of SG formation to the phosphorylation of the translation initiation factor eIF2α, and negative feedback via GADD34, a stress-induced subunit of protein phosphatase 1. We identify GADD34 messenger RNA levels as the molecular memory of the ISR that plays a central role in cell adaptation to acute and chronic stress.