Gait disturbances are common in the elderly. Cerebral small vessel disease, including white matter lesions and lacunars infarcts, is thought to disrupt white matter tracts that connect important motor regions, hence resulting in gait disturbances. Pathological studies have demonstrated abnormalities in white matter that may appear normal on brain imaging. The loss of integrity in such normal-appearing white matter may partly be due to small vessel disease and may play a role in causing gait disturbances. The white matter regions involved in these gait disturbances, both in white matter lesions and normal-appearing white matter, remain unclear. We, therefore, aimed to investigate the relation between the location of white matter lesions and gait using voxel-based morphometry analysis, as well as between white matter integrity and gait by applying tract-based spatial statistics to diffusion tensor imaging parameters. Magnetic resonance imaging was carried out on 429 individuals in the age range of 50 and 85 years, with cerebral small vessel disease without dementia or parkinsonism. Gait was assessed quantitatively. White matter lesions, especially in the centrum semiovale and periventricular frontal lobe, were related to a lower gait velocity, shorter stride length and broader stride width. Loss of white matter integrity, as indicated by a lower fractional anisotropy and higher mean diffusivity, in numerous regions was related to a lower gait performance. Most of these regions were located in the normal-appearing white matter. The strongest significant association was found in the corpus callosum, particularly the genu. Most of the associations in the normal-appearing white matter disappeared after controlling for white matter lesions and lacunar infarcts, except for some in the corpus callosum. In conclusion, our study showed that using a combination of voxel-based morphometry analysis of the white matter lesions and diffusion tensor imaging is of added value in investigating the pathophysiology of gait disturbances in subjects with small vessel disease. Our data demonstrate that, in elderly subjects with small vessel disease, widespread disruption of white matter integrity, predominantly in the normal-appearing white matter, is involved in gait disturbances. In particular, loss of fibres interconnecting bilateral cortical regions, especially the prefrontal cortex that is involved in cognitive control on motor performance, may be important. The most important mechanisms underlying affected normal-appearing white matter are probably a direct effect of small vessel disease or, indirectly, remote effects of white matter lesions and lacunar infarcts.