Dietary restriction (DR) during adulthood can greatly extend lifespan and improve metabolic health in diverse species. However, whether DR in mammals is still effective when applied for the first time at old age remains elusive. Here, we report results of a late-life DR switch experiment employing 800 mice, in which 24 months old female mice were switched from ad libitum (AL) to DR or vice versa. Strikingly, the switch from DR-to-AL acutely increases mortality, whereas the switch from AL-to-DR causes only a weak and gradual increase in survival, suggesting a memory of earlier nutrition. RNA-seq profiling in liver, brown (BAT) and white adipose tissue (WAT) demonstrate a largely refractory transcriptional and metabolic response to DR after AL feeding in fat tissue, particularly in WAT, and a proinflammatory signature in aged preadipocytes, which is prevented by chronic DR feeding. Our results provide evidence for a nutritional memory as a limiting factor for DR-induced longevity and metabolic remodeling of WAT in mammals.