Abstract:According to a classical view of face perception (Bruce and Young, 1986; Haxby et al., 2000), face identity and facial expression recognition are performed by separate neural substrates (ventral and lateral temporal face-selective regions, respectively). However, recent studies challenge this view, showing that expression valence can also be decoded from ventral regions (Skerry and Saxe, 2014; Li et al., 2019), and identity from lateral regions (Anzellotti and Caramazza, 2017). These findings could be reconciled with the classical view if regions specialized for one task (either identity or expression) contain a small amount of information for the other task (that enables above-chance decoding). In this case, we would expect representations in lateral regions to be more similar to representations in deep convolutional neural networks (DCNNs) trained to recognize facial expression than to representations in DCNNs trained to recognize face identity (the converse should hold for ventral regions). We tested this hypothesis by analyzing neural responses to faces varying in identity and expression. Representational dissimilarity matrices (RDMs) computed from human intracranial recordings (n = 11 adults; 7 females) were compared with RDMs from DCNNs trained to label either identity or expression. We found that RDMs from DCNNs trained to recognize identity correlated with intracranial recordings more strongly in all regions tested-even in regions classically hypothesized to be specialized for expression. These results deviate from the classical view, suggesting that face-selective ventral and lateral regions contribute to the representation of both identity and expression.SIGNIFICANCE STATEMENT Previous work proposed that separate brain regions are specialized for the recognition of face identity and facial expression. However, identity and expression recognition mechanisms might share common brain regions instead. We tested these alternatives using deep neural networks and intracranial recordings from face-selective brain regions. Deep neural networks trained to recognize identity and networks trained to recognize expression learned representations that correlate with neural recordings. Identity-trained representations correlated with intracranial recordings more strongly in all regions tested, including regions hypothesized to be expression specialized in the classical hypothesis. These findings support the view that identity and expression recognition rely on common brain regions. This discovery may require reevaluation of the roles that the ventral and lateral neural pathways play in processing socially relevant stimuli.

Concurrent emergence of view invariance, sensitivity to critical features, and identity face classification through visual experience: Insights from deep learning algorithms

Face recognition depends on specialized mechanisms tuned to view-invariant facial features: Insights from deep neural networks optimized for face or object recognition

Deep learning algorithms reveal a new visual-semantic representation of familiar faces in human perception and memory

Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation

Emerged human-like facial expression representation in a deep convolutional neural network

Deep Convolutional Neural Network Features and the Original Image

Exploring Features and Attributes in Deep Face Recognition Using Visualization Techniques

Intracranial Electroencephalography and Deep Neural Networks Reveal Shared Substrates for Representations of Face Identity and Expressions

Decoding Face Identity: A Reverse-Correlation Approach Using Deep Learning

Emergence of brain-like mirror-symmetric viewpoint tuning in convolutional neural networks

Advancements in Face Recognition: From Feature Extraction to Deep Learning Models and Integrated Biometric Solutions

The Face Module Emerged in a Deep Convolutional Neural Network Selectively Deprived of Face Experience

A Comprehensive Analysis of Deep Learning Based Representation for Face Recognition

Using deep neural networks to disentangle visual and semantic information in human perception and memory

Multidimensional Face Representation in a Deep Convolutional Neural Network Reveals the Mechanism Underlying AI Racism

Humans and deep networks largely agree on which kinds of variation make object recognition harder

Pursuing Face Identity From View-Specific Representation To View-Invariant Representation

Twin identification over viewpoint change: A deep convolutional neural network surpasses humans

Deep convolutional neural network with face identity recognition experience exhibits brain-like neural representations of personality traits

Understanding of Facial Features in Face Perception: Insights from Deep Convolutional Neural Networks

Deeper Interpretability of Deep Networks