Individual specialization underpins the division of labour within ant societies, but only in a small minority do morphological specialists, or physical castes, exist in the workforce. The genetic conditions that allow such castes to evolve are well understood, but the ecological pressures that select for them are not. We provide compelling evidence that the task of transporting novel prey selected for an exaggerated transport caste, or 'submajor', in the army ant Eciton burchellii. This species is the only Eciton that preys upon large arthropods as well as ants, the ancestral prey type, and by comparing load-transport among Eciton species and within E. burchellii, we show that this mixed diet significantly constrains transport efficiency. Crucially, however, we also show that E. burchellii submajors are highly specialized on transporting non-ant prey, and we demonstrate experientially that it is specifically this prey type that constrains prey-transport efficiency. Our study also suggests that phylogenetic constraints associated with the Eciton lifestyle intensified selection for the exaggerated submajor of E. burchellii. Thus, we propose that a novel task may only select for a special caste when phylogenetic constraints preclude the evolution of alternative solutions. This identifies a new and potentially general scenario for the evolution of physical castes.