Abstract:According to a classical view of face perception (Bruce and Young, 1986; Haxby et al., 2000), face identity and facial expression recognition are performed by separate neural substrates (ventral and lateral temporal face-selective regions, respectively). However, recent studies challenge this view, showing that expression valence can also be decoded from ventral regions (Skerry and Saxe, 2014; Li et al., 2019), and identity from lateral regions (Anzellotti and Caramazza, 2017). These findings could be reconciled with the classical view if regions specialized for one task (either identity or expression) contain a small amount of information for the other task (that enables above-chance decoding). In this case, we would expect representations in lateral regions to be more similar to representations in deep convolutional neural networks (DCNNs) trained to recognize facial expression than to representations in DCNNs trained to recognize face identity (the converse should hold for ventral regions). We tested this hypothesis by analyzing neural responses to faces varying in identity and expression. Representational dissimilarity matrices (RDMs) computed from human intracranial recordings (n = 11 adults; 7 females) were compared with RDMs from DCNNs trained to label either identity or expression. We found that RDMs from DCNNs trained to recognize identity correlated with intracranial recordings more strongly in all regions tested-even in regions classically hypothesized to be specialized for expression. These results deviate from the classical view, suggesting that face-selective ventral and lateral regions contribute to the representation of both identity and expression.SIGNIFICANCE STATEMENT Previous work proposed that separate brain regions are specialized for the recognition of face identity and facial expression. However, identity and expression recognition mechanisms might share common brain regions instead. We tested these alternatives using deep neural networks and intracranial recordings from face-selective brain regions. Deep neural networks trained to recognize identity and networks trained to recognize expression learned representations that correlate with neural recordings. Identity-trained representations correlated with intracranial recordings more strongly in all regions tested, including regions hypothesized to be expression specialized in the classical hypothesis. These findings support the view that identity and expression recognition rely on common brain regions. This discovery may require reevaluation of the roles that the ventral and lateral neural pathways play in processing socially relevant stimuli.

Functional Integration Of The Posterior Superior Temporal Sulcus Correlates With Facial Expression Recognition

The Structural and Functional Correlates of the Efficiency in Fearful Face Detection.

Neural Univariate Activity and Multivariate Pattern in the Posterior Superior Temporal Sulcus Differentially Encode Facial Expression and Identity

The Fusiform Face Area is Engaged in Holistic, Not Parts-Based, Representation of Faces.

Neuronal Function in the Cortical Face Perception Network

Functional Connectivity Pattern in the Core Face Network Reflects Different Mechanisms of Holistic Face Processing Measured by the Whole-Part Effect and Composite-Face Effect

The Hierarchical Structure of the Face Network Revealed by Its Functional Connectivity Pattern

A Study on Neural Mechanism of Face Processing Based on Fmri

Resting-state neural activity across face-selective cortical regions is behaviorally relevant.

The Neural Network for Face Recognition: Insights from an Fmri Study on Developmental Prosopagnosia.

Facial expression recognition: A meta-analytic review of theoretical models and neuroimaging evidence

Reexamining the neural network involved in perception of facial expression: A meta-analysis

Rostro-caudal organization of the human posterior superior temporal sulcus revealed by connectivity profiles.

A Study of Specific Neural Substrate for Face Processing

The Study on Functional Connectivity Between Frontal Eye Field and Other Brain Cortex in Visual Processing

The Preliminary Study of Facial Expression Recognition in Normal Adults by Fmri

Intracranial Electroencephalography and Deep Neural Networks Reveal Shared Substrates for Representations of Face Identity and Expressions

Functional MRI Study on Recognition of Facial Expression in Healthy Subjects

Neural Substrates for Explicit Recognition of Dynamic Facial Expression by Fmri

Processing communicative facial and vocal cues in the superior temporal sulcus

Decoding the temporal representation of facial expression in face-selective regions