Abstract:Although sensory processing is pivotal to nearly every theory of emotion, the evaluation of the visual input as 'emotional' (e.g. a smile as signalling happiness) has been traditionally assumed to take place in supramodal 'limbic' brain regions. Accordingly, subcortical structures of ancient evolutionary origin that receive direct input from the retina, such as the superior colliculus (SC), are traditionally conceptualized as passive relay centres. However, mounting evidence suggests that the SC is endowed with the necessary infrastructure and computational capabilities for the innate recognition and initial categorization of emotionally salient features from retinal information. Here, we built a neurobiologically inspired convolutional deep neural network (DNN) model that approximates physiological, anatomical and connectional properties of the retino-collicular circuit. This enabled us to characterize and isolate the initial computations and discriminations that the DNN model of the SC can perform on facial expressions, based uniquely on the information it directly receives from the virtual retina. Trained to discriminate facial expressions of basic emotions, our model matches human error patterns and above chance, yet suboptimal, classification accuracy analogous to that reported in patients with V1 damage, who rely on retino-collicular pathways for non-conscious vision of emotional attributes. When presented with gratings of different spatial frequencies and orientations never 'seen' before, the SC model exhibits spontaneous tuning to low spatial frequencies and reduced orientation discrimination, as can be expected from the prevalence of the magnocellular (M) over parvocellular (P) projections. Likewise, face manipulation that biases processing towards the M or P pathway affects expression recognition in the SC model accordingly, an effect that dovetails with variations of activity in the human SC purposely measured with ultra-high field functional magnetic resonance imaging. Lastly, the DNN generates saliency maps and extracts visual features, demonstrating that certain face parts, like the mouth or the eyes, provide higher discriminative information than other parts as a function of emotional expressions like happiness and sadness. The present findings support the contention that the SC possesses the necessary infrastructure to analyse the visual features that define facial emotional stimuli also without additional processing stages in the visual cortex or in 'limbic' areas. This article is part of the theme issue 'Cracking the laugh code: laughter through the lens of biology, psychology and neuroscience'.

Role of the Pulvinar in Visual Affective Scene Processing

Assessing the Role of the Pulvinar in Feature versus Spatial Attention Control using Deep Neural Networks

Decoding Multivoxel Representations of Affective Scenes in Retinotopic Visual Cortex

Beyond Auditory Relay: Dissecting the Inferior Colliculus’s Role in Sensory Prediction, Cognitive Decision-Making, and Reward Prediction

Adaptive Pulvinar Circuitry Supports Visual Cognition

Pulvinar Response Profiles and Connectivity Patterns to Object Domains

Human medial pulvinar is involved in face and tactile processing and dynamically adjusts its cortical connectivity based on ongoing stimulation

Decoding the Temporal Dynamics of Affective Scene Processing

Decoding Visual Feature Versus Visual Spatial Attention Control with Deep Neural Networks

The Pulvinar Regulates Information Transmission Between Cortical Areas Based on Attention Demands

Decoding fMRI Data: A Comparison Between Support Vector Machines and Deep Neural Networks

Sensory encoding of emotion conveyed by the face and visual context

Exploring distinct and joint contributions of the Locus Coeruleus and the Substantia Nigra/Ventral Tegmental Area complex to reward and valence processing using high-resolution fMRI

Decoding fMRI data with support vector machines and deep neural networks

A deep neural network model of the primate superior colliculus for emotion recognition

Dissociable Roles of Internal Feelings and Face Recognition Ability in Facial Expression Decoding.

Emergence of Emotion Selectivity in Deep Neural Networks Trained to Recognize Visual Objects

Insular dichotomy in the implicit detection of emotions in human faces

Alterations of the pulvinar in posterior cortical atrophy: A multimodal MRI study

Functional Imaging of the Human Lateral Geniculate Nucleus and Pulvinar

Multiple focal pulvinar projection fields in the macaque cortex