Abstract:According to a classical view of face perception (Bruce and Young, 1986; Haxby et al., 2000), face identity and facial expression recognition are performed by separate neural substrates (ventral and lateral temporal face-selective regions, respectively). However, recent studies challenge this view, showing that expression valence can also be decoded from ventral regions (Skerry and Saxe, 2014; Li et al., 2019), and identity from lateral regions (Anzellotti and Caramazza, 2017). These findings could be reconciled with the classical view if regions specialized for one task (either identity or expression) contain a small amount of information for the other task (that enables above-chance decoding). In this case, we would expect representations in lateral regions to be more similar to representations in deep convolutional neural networks (DCNNs) trained to recognize facial expression than to representations in DCNNs trained to recognize face identity (the converse should hold for ventral regions). We tested this hypothesis by analyzing neural responses to faces varying in identity and expression. Representational dissimilarity matrices (RDMs) computed from human intracranial recordings (n = 11 adults; 7 females) were compared with RDMs from DCNNs trained to label either identity or expression. We found that RDMs from DCNNs trained to recognize identity correlated with intracranial recordings more strongly in all regions tested-even in regions classically hypothesized to be specialized for expression. These results deviate from the classical view, suggesting that face-selective ventral and lateral regions contribute to the representation of both identity and expression.SIGNIFICANCE STATEMENT Previous work proposed that separate brain regions are specialized for the recognition of face identity and facial expression. However, identity and expression recognition mechanisms might share common brain regions instead. We tested these alternatives using deep neural networks and intracranial recordings from face-selective brain regions. Deep neural networks trained to recognize identity and networks trained to recognize expression learned representations that correlate with neural recordings. Identity-trained representations correlated with intracranial recordings more strongly in all regions tested, including regions hypothesized to be expression specialized in the classical hypothesis. These findings support the view that identity and expression recognition rely on common brain regions. This discovery may require reevaluation of the roles that the ventral and lateral neural pathways play in processing socially relevant stimuli.

A robust neural index of automatic generalization across variable natural views of familiar face identities

Mapping Face Recognition Information Use across Cultures.

Understanding human individuation of unfamiliar faces with oddball fast periodic visual stimulation and electroencephalography

An Iterative Approach for EEG based Rapid Face Search: A Refined Retrieval by Brain Computer Interfaces

Expertise for conspecific face individuation in the human brain

Neural discrimination of facial cues associated with trustworthiness in adults and 6-month-old infants as revealed by fast periodic visual stimulation.

Neural Correlates of Face Familiarity Perception

Rapid Face Recognition Based On Single-Trial Event-Related Potential Detection Over Multiple Brains

The temporal dynamics of familiar face recognition: Event-related brain potentials reveal the efficient activation of facial identity representations

Human Recognition Using Deep Neural Networks and Spatial Patterns of SSVEP Signals

A Rapid Face Recognition Bci System Using Single-Trial Erp

Characterizing the shared signals of face familiarity: Long-term acquaintance, voluntary control, and concealed knowledge

(Micro)saccade-related potentials during face recognition: A study combining EEG, eye-tracking, and deconvolution modeling

EEG-FRM: a neural network based familiar and unfamiliar face EEG recognition method

Cross-validating the electrophysiological markers of early face categorization

Combining Real‐Time Neuroimaging With Machine Learning to Study Attention to Familiar Faces During Infancy: A Proof of Principle Study

Intracranial Electroencephalography and Deep Neural Networks Reveal Shared Substrates for Representations of Face Identity and Expressions

Extensive Visual Training in Adulthood Reduces an Implicit Neural Marker of the Face Inversion Effect

Decoding face recognition abilities in the human brain

Single trial ERP amplitudes reveal the time course of acquiring representations of novel faces in individual participants

The effect of familiarity on behavioral oscillations in face perception