Depression is associated with markers of accelerated aging, but it is unclear how this relationship changes across the lifespan. We examined whether a brain-based measure of accelerated aging differed between depressed and never-depressed subjects across the adult lifespan and whether it was related to cognitive performance and disability. We applied a machine-learning approach that estimated brain age from structural MRI data in two depressed cohorts, respectively 170 midlife adults and 154 older adults enrolled in studies with common entry criteria. Both cohorts completed broad cognitive batteries and the older subgroup completed a disability assessment. The machine-learning model estimated brain age from MRI data, which was compared to chronological age to determine the brain-age gap (BAG; estimated age-chronological age). BAG did not differ between midlife depressed and nondepressed adults. Older depressed adults exhibited significantly higher BAG than nondepressed elders (Wald χ2 = 8.84, p = 0.0029), indicating a higher estimated brain age than chronological age. BAG was not associated with midlife cognitive performance. In the older cohort, higher BAG was associated with poorer episodic memory performance (Wald χ2 = 4.10, p = 0.0430) and, in the older depressed group alone, slower processing speed (Wald χ2 = 4.43, p = 0.0354). We also observed a statistical interaction where greater depressive symptom severity in context of higher BAG was associated with poorer executive function (Wald χ2 = 5.89, p = 0.0152) and working memory performance (Wald χ2 = 4.47, p = 0.0346). Increased BAG was associated with greater disability (Wald χ2 = 6.00, p = 0.0143). Unlike midlife depression, geriatric depression exhibits accelerated brain aging, which in turn is associated with cognitive and functional deficits.