Reference frames for spatial frequency in face representation differ in the temporal visual cortex and amygdala

Abstract

Social communication in nonhuman primates and humans is strongly affected by facial information from other individuals. Many cortical and subcortical brain areas are known to be involved in processing facial information. However, how the neural representation of faces differs across different brain areas remains unclear. Here, we demonstrate that the reference frame for spatial frequency (SF) tuning of face-responsive neurons differs in the temporal visual cortex and amygdala in monkeys. Consistent with psychophysical properties for face recognition, temporal cortex neurons were tuned to image-based SFs (cycles/image) and showed viewing distance-invariant representation of face patterns. On the other hand, many amygdala neurons were influenced by retina-based SFs (cycles/degree), a characteristic that is useful for social distance computation. The two brain areas also differed in the luminance contrast sensitivity of face-responsive neurons; amygdala neurons sharply reduced their responses to low luminance contrast images, while temporal cortex neurons maintained the level of their responses. From these results, we conclude that different types of visual processing in the temporal visual cortex and the amygdala contribute to the construction of the neural representations of faces.

Figure 1.

Visual stimuli. A, Original face images consisted of three different monkeys displaying three different facial expressions (open-mouth, neutral, and pout-lips). B, Bandpass filtered face images. Bandpass filters with different center image-based SFs were applied to an original face image (top left in A) in seven steps. C, An example of changes in stimulus size. Size manipulation was applied to a filtered face image (center in B) in five steps. Size manipulation was applied to all filtered face images; thus, a stimulus set consisted of 35 images.

Figure 2.

Tunings for image-based and retina-based SFs. A, B, Optimal stimuli at different sizes for hypothetical neurons tuned to image-based SFs (A) and retina-based SFs (B). These images were created by applying two-dimensional bandpass filters that share the same center image-based SF (A) or same center retina-based SF (B) across different sizes. C, D, Tuning curves for image-based SFs of hypothetical neurons tuned to image-based SFs (C) and to retina-based SFs (D). Curves with different colors correspond to tuning curves at different sizes. E, F, Response fields of neurons tuned to image-based SFs (E) and to retina-based SFs (F). The response magnitude is indicated by a color map.

Figure 3.

Recording sites. A, Lateral view of the right hemisphere of Monkey S. The vertical line is positioned at 25 mm anterior to the ear canal (A25). B, A coronal slice of magnetic resonance images at A25. The white trapezoid indicates the recording chamber. Dotted and dashed lines indicate electrode penetrations to the temporal cortex and the amygdala, respectively. C, A coronal section corresponding to a region surrounded by the black rectangle in B. D–F, Photographs of regions corresponding to the gray squares in C. Arrows indicate electric lesions made along the penetrations. G, Reconstructed recording sites in the right hemisphere of Monkey S. H, Hippocampus; L, lateral nucleus; B, basal nucleus; AB, accessory basal nucleus; C, central nucleus of amygdala; sts, superior temporal sulcus; amts, anterior middle temporal sulcus; rs, rhinal sulcus. The SI values did not differ in the lateral (n = 21) and basal nuclei (n = 6) of the amygdala in Monkey S (Mann–Whitney test; p = 0.66).

Figure 4.

Responses of face-responsive temporal cortex neurons to filtered face images of different sizes. A, B, Tuning curves of two different neurons for image-based SFs. Symbols and error bars show mean ± SEM. of firing rates during stimulus presentation. Solid curves indicate a set of Gaussian functions fitted to the data. Different symbols with different colors represent tuning curves obtained with five different stimulus sizes. Dotted lines indicate the mean firing rate during the fixation period before stimulus presentation. Preferred original face images were Monkey A displaying pout-lips (Fig. 1A, top right) in A and Monkey B displaying a neutral expression (Fig. 1A, middle center) in B. C, D, Response fields in a 2D plot of image-based SFs and stimulus sizes. The mean firing rates for each stimulus condition of the neurons in A and B are replotted in C and D, respectively. The data are linearly interpolated and represented by a color map.

Figure 5.

Responses of face-responsive amygdala neurons to filtered face images of different sizes. A–C, Tuning curves of three different neurons. Preferred original face images were Monkey C displaying open-mouth (Fig. 1A, bottom left) in A, Monkey A displaying open-mouth (Fig. 1A, top left) in B, and Monkey B displaying a neutral expression (Fig. 1A, middle center) in C. D–F, Response fields of the same neurons for combinations of image-based SFs and stimulus sizes. Other conventions are the same as in Figure 4.

Figure 6.

Tuning curves plotted with an axis of retina-based SFs. Responses of a temporal cortex neuron, shown in Figure 4A, and of an amygdala neuron, shown in Figure 5A, are replotted in A and B, respectively. Other conventions are the same as in Figure 4.

Figure 7.

Comparisons of effects of stimulus size on image-based SF tunings. A, C, Distribution histograms of the SI for face-responsive neurons in the temporal cortex (A; TC) and the amygdala (B; AM). Arrows indicate median values (0.10 for TC and 0.38 for AM). B, D, Averaged response fields across the temporal cortex neurons (B) and amygdala neurons (D). Solid lines represent contour lines.

Figure 8.

Temporal profiles of responses of face-responsive neurons. A, C, Average normalized peristimulus time histograms (PSTHs) of temporal cortex neurons (A) and amygdala neurons (C). We first made a PSTH with a bin width of 10 ms for each neuron across all of the trials for stimuli that elicited statistically significant responses. We then normalized the PSTH of each neuron by assigning its maximum to a value of 1 and the firing rate during the prestimulus period to a value of 0. For the average normalized PSTHs, we defined a bin as the onset of the responses where the value first exceeded a criterion (mean + 3 SD during the fixation period before stimulus presentation). The onset of the responses was in the 71 to 80 ms bin for both the temporal cortex and the amygdala. Shaded areas indicate mean ± SEM. Dotted lines indicate 70 ms after stimulus onset. B, D, Distribution histograms of the response latency for temporal cortex neurons (B) and amygdala neurons (D). Arrows indicate median values (138 and 126 ms for the temporal cortex and amygdala neurons, respectively). E, Relationship between the SI and the response latency for the temporal cortex neurons (filled circles) and the amygdala (open circles). There was no correlation between the two in either the temporal cortex (Spearman's rank correlation; r = 0.063, p = 0.67) or amygdala (r = −0.073, p = 0.64).

Figure 9.

Luminance contrast sensitivity of face-responsive neurons. A, An example of visual stimuli when testing for luminance contrast sensitivity. Total luminance contrast of an original face image (Fig. 1A, top left) was reduced in seven steps. B, C, Example responses of a temporal cortex neuron (B) and an amygdala neuron (C) to a series of contrast manipulated face images. Symbols and error bars indicate mean ± SEM. of firing rates during stimulus presentation. The neurons were tested by the stimulus set created from an original face image of Monkey A displaying a neutral expression (Fig. 1A, top center) in B, and Monkey A displaying pout-lips (Fig. 1A, top right) in C. D, Average normalized responses of temporal cortex neurons (n = 23; filled circles with a solid line) and amygdala neurons (n = 21; open circles with a dotted line). Symbols and error bars indicate mean ± SEM. of normalized mean firing rates across populations.