Modern single cell experiments have revealed unexpected heterogeneity in apparently functionally 'pure' cell populations. However, we are still lacking a conceptual framework to understand this heterogeneity. Here, we propose that cellular memories-changes in the molecular status of a cell in response to a stimulus, that modify the ability of the cell to respond to future stimuli-are an essential ingredient in any such theory. We illustrate this idea by considering a simple age-structured model of stem cell proliferation that takes account of mitotic memories. Using this model we argue that asynchronous mitosis generates heterogeneity that is central to stem cell population function. This model naturally explains why stem cell numbers increase through life, yet regenerative potency simultaneously declines.