The structural correlates of gender differences in visuospatial processing are essentially unknown. Our quantitative analysis of the cytoarchitecture of the human primary visual cortex [V1/Brodmann area 17 (BA17)], neighboring area V2 (BA18), and the cytoarchitectonic correlate of the motion-sensitive complex (V5/MT+/hOc5) shows that the visual areas are sexually dimorphic and that the type of dimorphism differs among the areas. Gender differences exist in the interhemispheric asymmetry of hOc5 volumes and in the right-hemispheric volumetric ratio of hOc5 to BA17, an area that projects to V5/MT+/hOc5. Asymmetry was also observed in the surface area of hOc5 but not in its cortical thickness. The differences give males potentially more space in which to process additional information, a finding consistent with superior male processing in particular visuospatial tasks, such as mental rotation. Gender differences in hOc5 exist with similar volume fractions of cell bodies, implying that, overall, the visual neural circuitry is similar in males and females.