Abstract:Recent neuroimaging studies have shown that the visual cortex plays an important role in representing the affective significance of visual input. The origin of these affect-specific visual representations is debated: they are intrinsic to the visual system versus they arise through reentry from frontal emotion processing structures such as the amygdala. We examined this problem by combining convolutional neural network (CNN) models of the human ventral visual cortex pre-trained on ImageNet with two datasets of affective images. Our results show that in all layers of the CNN models, there were artificial neurons that responded consistently and selectively to neutral, pleasant, or unpleasant images and lesioning these neurons by setting their output to zero or enhancing these neurons by increasing their gain led to decreased or increased emotion recognition performance respectively. These results support the idea that the visual system may have the intrinsic ability to represent the affective significance of visual input and suggest that CNNs offer a fruitful platform for testing neuroscientific theories. What is the role played by sensory cortices in assessing the emotional significance of sensory input? This question is attracting increasing research interest. Recent work has found affect-specific neural representations in visual cortex. The origins of these representations are debated. According to the reentry hypothesis, these representations result from reentrant feedback arising from anterior emotion processing structures such as the amygdala. An alternative hypothesis holds that sensory cortex may have the intrinsic capacity to represent the emotional qualities of sensory input. We examined this problem by utilizing the convolutional neural networks (CNNs) trained to recognized visual objects as computational models of the primate ventral visual system. Emotionally charged images were divided into three broad categories (pleasant, neutral and unpleasant) and presented to the CNNs. Responses of artificial neurons to these images were found to exhibit robust emotion selectivity. Importantly, enhancing the neurons that were selective for a given emotion led to the increased ability in recognizing that emotion, whereas lesioning these neurons led to the decrease in that ability. This research lends support to the notion that emotional perception might be an intrinsic property of the visual cortex. It also underscores the CNNs' value in examining neuroscientific theories.

Towards emotionally intelligent buildings: A Convolutional neural network based approach to classify human emotional experience in virtual built environments

Temporal Convolutional Network-Enhanced Real-Time Implicit Emotion Recognition with an Innovative Wearable fNIRS-EEG Dual-Modal System

Customized 2D CNN Model for the Automatic Emotion Recognition Based on EEG Signals

A 3D Convolutional Neural Network for Emotion Recognition Based on EEG Signals.

Emotion recognition using effective connectivity and pre-trained convolutional neural networks in EEG signals

Design and Implementation of an Emotion Analysis System Based on EEG Signals

A Model for EEG-Based Emotion Recognition: CNN-Bi-LSTM with Attention Mechanism

ResNet-50 based technique for EEG image characterization due to varying environmental stimuli

Hybrid deep models for parallel feature extraction and enhanced emotion state classification

Expert System for Smart Virtual Facial Emotion Detection Using Convolutional Neural Network

Can CNNs Accurately Classify Human Emotions? A Deep-Learning Facial Expression Recognition Study

Emotional Brain State Classification on fMRI Data Using Deep Residual and Convolutional Networks

EEG-based Emotion Recognition via Neural Architecture Search

Emotion Recognition from Spatio-Temporal Representation of EEG Signals via 3D-CNN with Ensemble Learning Techniques

Optimizing Residual Networks and VGG for Classification of EEG Signals: Identifying Ideal Channels for Emotion Recognition

1-D convolutional neural network based on the inner ear principle to automatically assess human’s emotional state

Decoding emotional responses to AI-generated architectural imagery

EEG-Based Emotion Recognition Using a 2D CNN with Different Kernels

Electroencephalograph-based emotion recognition using convolutional neural network without manual feature extraction

CNN and LSTM-Based Emotion Charting Using Physiological Signals

Emergence of Emotion Selectivity in Deep Neural Networks Trained to Recognize Visual Objects