Prevalence of selectivity for mirror-symmetric views of faces in the ventral and dorsal visual pathways

Abstract

Although the ability to recognize faces and objects from a variety of viewpoints is crucial to our everyday behavior, the underlying cortical mechanisms are not well understood. Recently, neurons in a face-selective region of the monkey temporal cortex were reported to be selective for mirror-symmetric viewing angles of faces as they were rotated in depth (Freiwald and Tsao, 2010). This property has been suggested to constitute a key computational step in achieving full view-invariance. Here, we measured functional magnetic resonance imaging activity in nine observers as they viewed upright or inverted faces presented at five different angles (-60, -30, 0, 30, and 60°). Using multivariate pattern analysis, we show that sensitivity to viewpoint mirror symmetry is widespread in the human visual system. The effect was observed in a large band of higher order visual areas, including the occipital face area, fusiform face area, lateral occipital cortex, mid fusiform, parahippocampal place area, and extending superiorly to encompass dorsal regions V3A/B and the posterior intraparietal sulcus. In contrast, early retinotopic regions V1-hV4 failed to exhibit sensitivity to viewpoint symmetry, as their responses could be largely explained by a computational model of low-level visual similarity. Our findings suggest that selectivity for mirror-symmetric viewing angles may constitute an intermediate-level processing step shared across multiple higher order areas of the ventral and dorsal streams, setting the stage for complete viewpoint-invariant representations at subsequent levels of visual processing.

Figure 1.

Stimuli. The stimuli included five different viewpoints (−60, −30, 0, 30, and 60°, upper row) of six different individuals (lower row).

Figure 2.

Control for low-level confounds. a, To exclude the possibility that low-level features of the stimuli would already lead to patterns of viewpoint mirror symmetry, a biologically realistic model of V1 simple cells was implemented (see Materials and Methods for details). As shown on an exemplary face on the right, the stimuli were spatially filtered (foveated) to account for differences in visual accuracy. The size of the face is proportional to the size of the Gabor filters used in the model. b, The V1 model responses to the standard FaceGen stimuli, as shown on the left, showed increased correlations for mirror-symmetric head orientations. This low-level confound was overcome by the addition of structured hair (shown on the right). c, The low-level similarity tuningcurves, as estimated from the model. The red “x” marks the mirror-symmetric viewpoint.

Figure 3.

Effect estimates for low-level similarity and viewpoint symmetry. a, In a given ROI, the effects of low-level similarity were estimated by correlating the upper triangle of the empirical correlation matrices of both hemispheres with a model of low-level similarity, derived from the output of a computational V1 model (shown left). The effects of viewpoint symmetry, which predict higher correlation values for viewpoints with mirror-symmetric viewing angles, were estimated based on the partial correlation between the empirical correlation matrix and the viewpoint-symmetry model (right), after first regressing out the effects of low-level similarity. b, Visualization of the average correlation matrix for the ROIs. Please note that we estimated the effect sizes for every subject individually and not based on these averages. c, The average effect sizes of low-level similarity in the different ROIs (error bars indicate SEM). All regions show significant effects. d, Average effect size of viewpoint symmetry. While higher level ROIs show significant effects of viewpoint symmetry, the early and intermediate-level areas V1–hV4 do not (see text for details and p values).

Figure 4.

a, b, Effects of low-level similarity (a) and viewpoint symmetry (b) during the processing of inverted faces. As in the upright condition, all areas show significant effects of low-level similarity, whereas only higher level areas show robust effects of viewpoint symmetry.

Figure 5.

Principal component analyses. a, When projected into two dimensions, the similarity of the correlation matrices of the different ROIs can be visualized. The first component, which already explains 81.8% variance, shows a clear separation between ROIs with and without effects of viewpoint symmetry (the second component explains an additional 13.9% of the variance). b, The resulting first component when computing a PCA directly on the entries of the correlation matrices. The component exhibits large weights in the two diagonals, in direct agreement with the effects of low-level similarity, and viewpoint symmetry.

Figure 6.

Searchlight results. Clusters of significant low-level similarity and viewpoint symmetry across subjects on the inflated (top) and flattened (bottom) standard brain. The light-blue line delineates regions showing significant effects of low-level similarity. Regions of significant viewpoint symmetry are marked in hot colors. They form a band of higher order visual regions, which excludes more posterior (early and intermediate-level) visual areas and more anterior areas. The delineated ROIs are from the localizer results of a representative subject (M051).