Abstract:According to a classical view of face perception (Bruce and Young, 1986; Haxby et al., 2000), face identity and facial expression recognition are performed by separate neural substrates (ventral and lateral temporal face-selective regions, respectively). However, recent studies challenge this view, showing that expression valence can also be decoded from ventral regions (Skerry and Saxe, 2014; Li et al., 2019), and identity from lateral regions (Anzellotti and Caramazza, 2017). These findings could be reconciled with the classical view if regions specialized for one task (either identity or expression) contain a small amount of information for the other task (that enables above-chance decoding). In this case, we would expect representations in lateral regions to be more similar to representations in deep convolutional neural networks (DCNNs) trained to recognize facial expression than to representations in DCNNs trained to recognize face identity (the converse should hold for ventral regions). We tested this hypothesis by analyzing neural responses to faces varying in identity and expression. Representational dissimilarity matrices (RDMs) computed from human intracranial recordings (n = 11 adults; 7 females) were compared with RDMs from DCNNs trained to label either identity or expression. We found that RDMs from DCNNs trained to recognize identity correlated with intracranial recordings more strongly in all regions tested-even in regions classically hypothesized to be specialized for expression. These results deviate from the classical view, suggesting that face-selective ventral and lateral regions contribute to the representation of both identity and expression.SIGNIFICANCE STATEMENT Previous work proposed that separate brain regions are specialized for the recognition of face identity and facial expression. However, identity and expression recognition mechanisms might share common brain regions instead. We tested these alternatives using deep neural networks and intracranial recordings from face-selective brain regions. Deep neural networks trained to recognize identity and networks trained to recognize expression learned representations that correlate with neural recordings. Identity-trained representations correlated with intracranial recordings more strongly in all regions tested, including regions hypothesized to be expression specialized in the classical hypothesis. These findings support the view that identity and expression recognition rely on common brain regions. This discovery may require reevaluation of the roles that the ventral and lateral neural pathways play in processing socially relevant stimuli.

Shape-invariant encoding of dynamic primate facial expressions in human perception

Neuromuscular Control of the Face-Head-Neck Biomechanical Complex With Learning-Based Expression Transfer From Images and Videos

EEG correlates of static and dynamic face perception: the role of naturalistic motion

Deep learning detects subtle facial expressions in a multilevel society primate

Equivalent processing of facial expression and identity by macaque visual system and task-optimized neural network

A Naturalistic Dynamic Monkey Head Avatar Elicits Species-Typical Reactions and Overcomes the Uncanny Valley

The Role of Expression and Identity in the Face-Selective Responses of Neurons in the Temporal Visual Cortex of the Monkey

A Novel Dynamic Model Capturing Spatial and Temporal Patterns for Facial Expression Analysis

MacAction: Realistic 3D macaque body animation based on multi-camera markerless motion capture

A Deep Learning Perspective on the Origin of Facial Expressions

Recognizing facial expressions of emotion amid noise: A dynamic advantage

The Brain Computes Dynamic Facial Movements for Emotion Categorization Using a Third Pathway

Neurophysiological Mechanisms Underlying Face Processing Within and Beyond the Temporal Cortical Visual Areas.

Humans and marmosets share similar face recognition signatures in shape-based visual face discrimination behavior

Real Face Value: The Processing of Naturalistic Facial Expressions in the Macaque Inferior Temporal Cortex

Quantifying human sensitivity to spatio-temporal information in dynamic faces

A deep neural network model of the primate superior colliculus for emotion recognition

Intracranial Electroencephalography and Deep Neural Networks Reveal Shared Substrates for Representations of Face Identity and Expressions

Unveiling the human-like similarities of automatic facial expression recognition: An empirical exploration through explainable ai

A System View of the Recognition and Interpretation of Observed Human Shape, Pose and Action

Facial Expression Retargeting from Human to Avatar Made Easy