Neural tuning for face wholes and parts in human fusiform gyrus revealed by FMRI adaptation

Abstract

Although the right fusiform face area (FFA) is often linked to holistic processing, new data suggest this region also encodes part-based face representations. We examined this question by assessing the metric of neural similarity for faces using a continuous carryover functional MRI (fMRI) design. Using faces varying along dimensions of eye and mouth identity, we tested whether these axes are coded independently by separate part-tuned neural populations or conjointly by a single population of holistically tuned neurons. Consistent with prior results, we found a subadditive adaptation response in the right FFA, as predicted for holistic processing. However, when holistic processing was disrupted by misaligning the halves of the face, the right FFA continued to show significant adaptation, but in an additive pattern indicative of part-based neural tuning. Thus this region seems to contain neural populations capable of representing both individual parts and their integration into a face gestalt. A third experiment, which varied the asymmetry of changes in the eye and mouth identity dimensions, also showed part-based tuning from the right FFA. In contrast to the right FFA, the left FFA consistently showed a part-based pattern of neural tuning across all experiments. Together, these data support the existence of both part-based and holistic neural tuning within the right FFA, further suggesting that such tuning is surprisingly flexible and dynamic.

Fig. 1.

Methods for the experiments. A: 4 endpoint faces (green boxes) varying along eye and mouth dimensions were morphed to create the experimental stimuli, which have equal 50% differences in both eye and mouth dimensions. Signal recovery in functional MRI (fMRI) was measured for stimulus transitions either along a single axis (e.g., mouth only), so-called “pure” transitions (blue), or “composite” changes along both eye and mouth axes (red). Because the original stimulus images used in the experiment were not approved for publication, example faces are shown in the figure. B: sample fMRI sequence. The measure of interest was the differential adaptation of the fMRI response for Pure vs. Composite stimulus transitions from the previous face. Subjects were not explicitly informed of the stimulus space arrangement or the transitions of interest and were instructed to monitor for the appearance of a target face. C: analyses focused on the fusiform face area (FFA) in left and right hemispheres, functionally defined in each subject from an independent localizer scan. These maps reflect the overlap of FFA across subjects (n = 16), shown atop the average of the registered anatomical images across subjects.

Fig. 2.

Predicted and measured neural tuning for experiment 1 (whole faces). A: because of the manner in which the stimulus spaces are constructed, different response patterns are predicted for Euclidean vs. “city block” metrics of distance, associated, respectively, with holistic and part-based processing. Left: neural tuning to the entire face will result in a subadditive, Euclidean distance between composite faces, with greater fMRI signal recovery for Pure (red) relative to Composite (blue) stimulus transitions. Right: conversely, tuning for independent parts is associated with additive part changes, resulting in equal recovery for Pure and Composite stimulus transitions. B: measured fMRI responses show that, although tuning in the left FFA is part-based, the right FFA is holistic. Error bars in this and all subsequent graphs reflect SE.

Fig. 3.

Measured neural tuning for experiment 2 (misaligned faces). To assess whether the right FFA can also display significant neural tuning for parts, we replicated the 1st experiment with horizontally misaligned faces, known to disrupt holistic processing. A: sample stimulus. All other experimental procedures and analyses remained the same as experiment 1. B: the right FFA continued to show significant fMRI signal recovery for Pure and Composite stimulus transitions relative to identical repetitions, but there was no significant difference between Pure and Composite transitions. Together, these data suggest that the right FFA contains neural populations tuned to parts and wholes.

Fig. 4.

Predicted and measured neural tuning for experiment 3 (asymmetric space). This experiment was designed to provide a positive measure of part-based neural tuning as a control for the previous 2 experiments. A: the asymmetric stimulus space has an 87% change in 1 feature along the major axis and a 50% change in the other feature on the minor axis. Because of this directionality, there are 4 possible versions of the asymmetric space (k1–k4); subjects were exposed to all 4 versions. B: the values of the asymmetric stimulus space were chosen to produce opposite predictions to those for the original face stimulus set. Left: if tuning is holistic, responses to Pure and Composite stimulus transitions will be equivalent in this space. Right: because of the asymmetric arrangement of the space, additive part-based tuning would result in greater recovery for Composite stimulus transitions. C: measured fMRI signal recovery for the asymmetric space. Although the left FFA response remains part-based, the right FFA response unexpectedly also displays part-based neural tuning.