Although the adult brain is considered to be fully developed and stable until senescence when its size steadily decreases, such stability seems at odds with continued human (intellectual) development throughout life. Moreover, although variation in human brain size is highly heritable, we do not know the extent to which genes contribute to individual differences in brain plasticity. In this longitudinal magnetic resonance imaging study in twins, we report considerable thinning of the frontal cortex and thickening of the medial temporal cortex with increasing age and find this change to be heritable and partly related to cognitive ability. Specifically, adults with higher intelligence show attenuated cortical thinning and more pronounced cortical thickening over time than do subjects with average or below average IQ. Genes influencing variability in both intelligence and brain plasticity partly drive these associations. Thus, not only does the brain continue to change well into adulthood, these changes are functionally relevant because they are related to intelligence.