Brain atrophy in multiple sclerosis (MS) is thought to reflect irreversible tissue damage leading to persistent clinical deficit. Little is known about the rate of atrophy in specific brain regions in relation to specific clinical deficits. We determined the displacement of the brain surface between two T1-weighted MRI images obtained at baseline and after a median follow-up time of 2.2 years for 79 recently diagnosed, mildly disabled MS patients. Voxel- and cluster-wise permutation-based statistics were used to identify brain regions in which atrophy development was significantly related to Expanded Disability Status Scale (EDSS), Timed Walk Test (TWT), Paced Auditory Serial Addition Test (PASAT) and 9-Hole Peg Test (HPT). Clusters were considered significant at a corrected cluster-wise p-value of 0.05. Worse EDSS change-score and worse follow-up EDSS were related to atrophy development of periventricular and brainstem regions and right-sided parietal, occipital and temporal regions. Worse PASAT at follow-up was significantly related to atrophy of the ventricles. A worse TWT change-score and worse follow-up TWT were exclusively related to atrophy around the ventricles and of the brainstem. Worse HPT change-score and worse follow-up HPT of either arm were significantly related to the atrophy of widely distributed peripheral regions, as well as atrophy of periventricular and brainstem regions. Our findings suggest that decline in ambulatory function is related to atrophy of central brain regions exclusively, whereas decline in neurologically more complex tasks for coordinated hand function is related to atrophy of both central and peripheral brain regions.