Exercise and cardiorespiratory (CR) fitness may moderate age-related regional brain changes in nondemented (ND) older adults. The relationship of fitness to Alzheimer disease (AD)-related brain change is understudied, particularly in the hippocampus, which is disproportionately affected in early AD. The role of apolipoprotein E4 (apoE4) genotype in modulating this relationship is also unknown. ND (n=56) and early-stage AD patients (n=61) over the age of 65 years had magnetic resonance imaging and CR fitness assessments. Voxel-based morphometry techniques were used to identify AD-related atrophy. We analyzed the relationship of CR fitness with white and gray matter within groups, assessed fitness-related brain volume change in areas most affected by AD-related atrophy, and then analyzed differential fitness-brain relationships between apoE4 carriers. Atrophy was present in the medial temporal, temporal, and parietal cortices in patients with mild AD. There was a significant positive correlation of CR fitness with parietal and medial temporal volume in AD patients. ND patients did not have a significant relationship between brain volume and CR fitness in the global or small volume correction analyses. There was not a significant interaction for fitness x apoE4 genotype in either group. In early-stage AD, CR fitness is associated with regional brain volumes in the medial-temporal and parietal cortices suggesting that maintaining CR fitness may modify AD-related brain atrophy.