The modular construction of many plants and animals defies conventional approaches to the study of life histories and population dynamics. An important complication of modular construction is that individuals can rapidly decrease in size when some modules are removed or die or when an individual fragments. Most attempts to describe life histories and population dynamics of modular organisms classify individuals according to their size. This approach relies on the fundamental assumption that fragmentation and module loss have no consequences for an individual apart from a simple decrease in size. Here we experimentally test this assumption. Using a modular marine invertebrate, the encrusting bryozoan Watersipora subtorquata, as a model species, we manipulated colony size and then assessed performance against three potential explanatory models based on size, age, and damage. In a second experiment we disrupted the internal modular demography of colonies to determine whether the performance of a fragment is influenced by the type of modules that remain. Finally, we investigated how constraints on growth in modular organisms uniquely influence growth after module loss. We found that single-state variables such as size or age do not describe performance in our species. Internal constraints substantially reduce growth after a decrease in size, and the age of modules that remain determines the timing of reproductive onset and fecundity. A knowledge of the size history of individuals, including any decreases in size, is necessary to accurately describe life histories and population dynamics in this modular organism. Our results have major consequences for established methods for modeling the demography of modular organisms.