This study examined how age and education influence the relationship between neuropsychological test scores and brain structure in demographically diverse older adults spanning the range from normal cognition to dementia. A sample of 351 African Americans, 410 Hispanics, and 458 Whites underwent neuropsychological testing. Volumetric magnetic resonance imaging (MRI) measures of total brain, white matter hyperintensity, and hippocampus were available for 79 African Americans, 102 Hispanics, and 134 Whites. The authors used latent variable modeling to examine effects of age, education, and brain volumes on test scores and determine how much variance brain volumes explained in unadjusted and age- and education-adjusted scores. Age adjustment resulted in weaker relationships of test scores with MRI variables; adjustment for ethnicity yielded stronger relationships. Education adjustment increased relationships with MRI variables in the combined sample and Hispanics, made no difference in Whites, but decreased some associations in African Americans. Results suggest that demographic adjustment is beneficial when demographic variables are strongly related to test scores independent of measures of brain structure, but adjustment has negative consequences when effects of demographic characteristics are mediated by brain structure.