Tropical forests include a diversity of habitats, which has led to specialization in plants. Near Iquitos, in the Peruvian Amazon, nutrient-rich clay forests surround nutrient-poor white-sand forests, each harboring a unique composition of habitat specialist trees. We tested the hypothesis that the combination of impoverished soils and herbivory creates strong natural selection for plant defenses in white-sand forest, while rapid growth is favored in clay forests. Recently, we reported evidence from a reciprocal-transplant experiment that manipulated the presence of herbivores and involved 20 species from six genera, including phylogenetically independent pairs of closely related white-sand and clay specialists. When protected from herbivores, clay specialists exhibited faster growth rates than white-sand specialists in both habitats. But, when unprotected, white-sand specialists outperformed clay specialists in white-sand habitat, and clay specialists outperformed white-sand specialists in clay habitat. Here we test further the hypothesis that the growth defense trade-off contributes to habitat specialization by comparing patterns of growth, herbivory, and defensive traits in these same six genera of white-sand and clay specialists. While the probability of herbivore attack did not differ between the two habitats, an artificial defoliation experiment showed that the impact of herbivory on plant mortality was significantly greater in white-sand forests. We quantified the amount of terpenes, phenolics, leaf toughness, and available foliar protein for the plants in the experiment. Different genera invested in different defensive strategies, and we found strong evidence for phylogenetic constraint in defense type. Overall, however, we found significantly higher total defense investment for white-sand specialists, relative to their clay specialist congeners. Furthermore, herbivore resistance consistently exhibited a significant trade-off against growth rate in each of the six phylogenetically independent species-pairs. These results confirm theoretical predictions that a trade-off exists between growth rate and defense investment, causing white-sand and clay specialists to evolve divergent strategies. We propose that the growth-defense trade-off is universal and provides an important mechanism by which herbivores govern plant distribution patterns across resource gradients.