The purpose of this study was to test the hypothesis that higher levels of systemic inflammation in a community sample of non-demented subjects older than seventy years of age are associated with reduced diffusion anisotropy in brain white matter and lower cognition. Ninety-five older persons without dementia underwent detailed clinical and cognitive evaluation and magnetic resonance imaging, including diffusion tensor imaging. Systemic inflammation was assessed with a composite measure of commonly used circulating inflammatory markers (C-reactive protein and tumor necrosis factor-alpha). Tract-based spatial statistics analyses demonstrated that diffusion anisotropy in the body and isthmus of the corpus callosum was negatively correlated with the composite measure of systemic inflammation, controlling for demographic, clinical and radiologic factors. Visuospatial ability was negatively correlated with systemic inflammation, and diffusion anisotropy in the body and isthmus of the corpus callosum was shown to mediate this association. The findings of the present study suggest that higher levels of systemic inflammation may be associated with lower microstructural integrity in the corpus callosum of non-demented elderly individuals, and this may partially explain the finding of reduced higher-order visual cognition in aging.