Currently, no reliable predictors of cognitive impairment in Parkinson's disease exist. We hypothesized that microstructural changes at grey matter T1-weighted MRI and diffusion tensor imaging in the cholinergic system nuclei and associated limbic pathways underlie cognitive impairment in Parkinson's disease. We performed a cross-sectional comparison between patients with Parkinson's disease with and without cognitive impairment. We also performed a longitudinal 36-month follow-up study of cognitively intact Parkinson's disease patients, comparing patients who remained cognitively intact to those who developed cognitive impairment. Patients with Parkinson's disease with cognitive impairment showed lower grey matter volume and increased mean diffusivity in the nucleus basalis of Meynert, compared to patients with Parkinson's disease without cognitive impairment. These results were confirmed both with region of interest and voxel-based analyses, and after partial volume correction. Lower grey matter volume and increased mean diffusivity in the nucleus basalis of Meynert was predictive for developing cognitive impairment in cognitively intact patients with Parkinson's disease, independent of other clinical and non-clinical markers of the disease. Structural and microstructural alterations in entorhinal cortex, amygdala, hippocampus, insula, and thalamus were not predictive for developing cognitive impairment in Parkinson's disease. Our findings provide evidence that degeneration of the nucleus basalis of Meynert precedes and predicts the onset of cognitive impairment, and might be used in a clinical setting as a reliable biomarker to stratify patients at higher risk of cognitive decline.