We investigated the relationship between temperamental predisposition and brain structure by using a standard questionnaire and high-resolution T1-weighted magnetic resonance image (MRI) in normal young volunteers. Fifty-six subjects completed the Japanese version of the Temperament and Character Inventory (TCI, 125 items) and underwent an MRI acquisition of the brain. The gray matter (GM) was extracted from the whole brain image of the subjects and normalized to the standard brain template using statistical parametric mapping and the optimized voxel-based morphometry (VBM) method. When the score on the harm avoidance (HA) subscale was used as a dependent variable, the multiple regression analysis revealed that the HA score positively correlated with the volume of the part of left amygdala. The region-of-interest analysis showed that the correlation was significant in the female subjects but not in the male subjects. The correlation was significant even after the effects of age, depression score, and total GM volume were taken into account. The differential correlation between the sexes may be caused by differences in hormonal condition and the vulnerability of women to socio-psychological stress. In addition, the novelty seeking (NS) score positively correlated with the GM of the left middle frontal gyrus. The volume of the tail of the right caudate nucleus positively correlated with the reward dependence (RD) score. With regard to the NS and RD scores, no significant sex difference was observed in the correlation. These results indicate that the temperamental traits measured using the questionnaire may have a morphological basis in the human brain.