The temperature-size rule, a form of phenotypic plasticity in which decreased temperature increases final size, is one of the most widespread patterns in biology, particularly for ectotherms. Identifying the environmental conditions in which this pattern is reversed is key to understanding the generality of the rule. We use wild and domesticated populations of the tobacco hornworm Manduca sexta and the natural host plants of this species to explore the consequences of resource quality for the temperature-size rule. Manduca sexta reared on a high-quality host, tobacco (Nicotiana tabacum), followed the temperature-size rule, with larger final sizes at lower temperatures. In contrast, M. sexta reared on a low-quality host, devil's claw (Proboscidea louisianica), showed the reverse response. Wild and domesticated M. sexta exhibited qualitatively similar responses. Survival, growth and development rates, fecundity, and final size decreased with decreasing temperature in M. sexta reared on devil's claw. We propose that the reversal of the temperature-size rule results from the stressful combination of low temperatures and low dietary quality. Such reversals may impact seasonal and geographic patterns of host use in Manduca and other systems. Our results suggest that the temperature-size rule occurs for a restricted range of nonstressful environmental conditions, limiting the robustness of this widespread pattern of phenotypic plasticity.