The social-cognitive deficits associated with several neurodevelopmental and neuropsychiatric disorders have been linked to structural and functional brain anomalies. Given the recent appreciation for quantitative approaches to behavior, in this study we examined the brain-behavior links in social cognition in healthy young adults from a quantitative approach. Twenty-two participants were administered quantitative measures of social cognition, including the social responsiveness scale (SRS), the empathizing questionnaire (EQ) and the systemizing questionnaire (SQ). Participants underwent a structural, 3-T magnetic resonance imaging (MRI) procedure that yielded both volumetric (voxel count) and asymmetry indices. Model fitting with backward elimination revealed that a combination of cortical, limbic and striatal regions accounted for significant variance in social behavior and cognitive styles that are typically associated with neurodevelopmental and neuropsychiatric disorders. Specifically, as caudate and amygdala volumes deviate from the typical R > L asymmetry, and cortical gray matter becomes more R > L asymmetrical, overall SRS and Emotion Recognition scores increase. Social Avoidance was explained by a combination of cortical gray matter, pallidum (rightward asymmetry) and caudate (deviation from rightward asymmetry). Rightward asymmetry of the pallidum was the sole predictor of Interpersonal Relationships and Repetitive Mannerisms. Increased D-scores on the EQ-SQ, an indication of greater systemizing relative to empathizing, was also explained by deviation from the typical R > L asymmetry of the caudate.These findings extend the brain-behavior links observed in neurodevelopmental disorders to the normal distribution of traits in a healthy sample.