The neural mechanisms of face processing: cells, areas, networks, and models

Abstract

Since its discovery, the face-processing network in the brain of the macaque monkey has emerged as a model system that allowed for major neural mechanisms of face recognition to be identified - with implications for object recognition at large. Populations of face cells encode faces through broad tuning curves, whose shapes change over time. Face representations differ qualitatively across faces areas, and we not only understand the global organization of these specializations, but also some of the transformations between face areas, both feed-forward and feed-back, and the computational principles behind face representations and transformations. Facial information is combined with physical features and mnemonic features in extensions of the core network, which forms an early part of the primate social brain.

Fig. 1

Top: Schematic side view of macaque brain with seven areas of face-selective cortex (red) in the temporal lobe together with connectivity graph (orange). Face areas are named based on their anatomical location: AF, anterior fundus; AL, anterior lateral; AM, anterior medial; MF, middle fundus; ML; middle lateral; PL, posterior lateral, and have been found to be directly connected to each other to form a face-processing network. Face-motion area MD, medial dorsal, is shown as well. Its connectivity with the other temporal lobe face areas is currently unknown. Middle: Classical view of the network operating in a feed-forward hierarchy. Bottom: Quantification of population tuning to head orientation and identity by three tuning coefficients (arbitrary units): View specificity and mirror symmetry describe the shape of tuning to head orientation, identity selectivity to identity across all head orientations. Population activity in ML is dominated by view specific representations. This tuning is still found in AL, but here a new quality emerges, mirror-symmetric tuning to head orientation. In AM, tuning to head orientation is substantially reduced, and identity selectivity dominates.

Fig. 2A

Schematic illustration of two alternative hypotheses about the function of face-processing pathways: The recognition or classification hypothesis (top) and the inverse-graphics or inference network hypothesis (bottom). The first hypothesis, currently dominant in AI and neuroscience, is that perception is best approached using neural networks optimized for classification, trained to recognize or distinguish object or face identities. The second hypothesis posits that face perception in the brain is best understood in terms of an inference network that inverts a causal generative model.

Fig. 2B

Top: The inference network (EIG, efficient inverse graphics) inverts a generative model using a cascade of deep neural networks with intermediate steps corresponding to processing stages of the ventral object recognition stream and face areas. Layer f3, the top convolutional layer, corresponds to face areas ML/MF, f4, the first fully connected layer, to area AL, and f6, the second fully connected layer, to face area AM. Bottom: Tuning coefficients in EIG layers f3, f4, and f5, are very similar to and highly correlated with those in MF/ML, AL, and AM (Fig. 1), respectively.

Figure 3A

Top: Bottom: Stimulus-aligned fMRI time courses within face area ML (left), TP, and PR (right) during the presentation of familiar faces (red), nonfamiliar faces (blue), and objects (gray). Percent signal change shown. Error bars represent standard error. Sigmoidal functions Naka-Rushton function fit to mean time courses are shown.

Fig. 3B

There are at least two specific additions to the core face-processing system, one for the processing of face motion and the other for the processing of familiar faces.