The California Verbal Learning Test (CVLT) is recognized as a standard clinical tool for assessing episodic memory difficulties in multiple sclerosis (MS), but its neural correlates have not yet been examined in detail in this patient population. We combined neuropsychological examination and diffusion tensor imaging (DTI) analysis in a group of MS patients (N = 50) and demographically matched healthy participants (N = 20). We investigated the degree of impairment of the uncinate fascicle (UF), the superior longitudinal fascicle (SLF), the fornix (FX) and the cingulum (CG). The patients were impaired on all CVLT parameters and the DTI parameters correlated moderately with disease-related variables. Regression analyses in the complete study sample showed that CVLT learning scores correlated with impairment of the right UF. This association reached marginal significance in the patient sample. In contrast to other studies claiming retrieval deficits, our results suggest that encoding and consolidation deficits may play a major role in verbal memory impairments in MS. The findings also provide evidence for an association between degree of myelination of prefrontal fibre pathways and encoding efficiency. Finally, DTI-derived measurements appear to reflect disease progression in MS. The results are discussed in light of functional MRI studies investigating compensatory brain activity during cognitive processing in MS.