Abstract:Recent neuroimaging studies have shown that the visual cortex plays an important role in representing the affective significance of visual input. The origin of these affect-specific visual representations is debated: they are intrinsic to the visual system versus they arise through reentry from frontal emotion processing structures such as the amygdala. We examined this problem by combining convolutional neural network (CNN) models of the human ventral visual cortex pre-trained on ImageNet with two datasets of affective images. Our results show that in all layers of the CNN models, there were artificial neurons that responded consistently and selectively to neutral, pleasant, or unpleasant images and lesioning these neurons by setting their output to zero or enhancing these neurons by increasing their gain led to decreased or increased emotion recognition performance respectively. These results support the idea that the visual system may have the intrinsic ability to represent the affective significance of visual input and suggest that CNNs offer a fruitful platform for testing neuroscientific theories. What is the role played by sensory cortices in assessing the emotional significance of sensory input? This question is attracting increasing research interest. Recent work has found affect-specific neural representations in visual cortex. The origins of these representations are debated. According to the reentry hypothesis, these representations result from reentrant feedback arising from anterior emotion processing structures such as the amygdala. An alternative hypothesis holds that sensory cortex may have the intrinsic capacity to represent the emotional qualities of sensory input. We examined this problem by utilizing the convolutional neural networks (CNNs) trained to recognized visual objects as computational models of the primate ventral visual system. Emotionally charged images were divided into three broad categories (pleasant, neutral and unpleasant) and presented to the CNNs. Responses of artificial neurons to these images were found to exhibit robust emotion selectivity. Importantly, enhancing the neurons that were selective for a given emotion led to the increased ability in recognizing that emotion, whereas lesioning these neurons led to the decrease in that ability. This research lends support to the notion that emotional perception might be an intrinsic property of the visual cortex. It also underscores the CNNs' value in examining neuroscientific theories.

Linking Neural Activity to Image Stimuli Through Convolutional Neural Networks: A Methodology

Detecting Brain Activations by Constrained Non-Negative Matrix Factorization from Task-Related BOLD Fmri

Exploring Brain Effective Connectivity in Visual Perception Using a Hierarchical Correlation Network.

Bridging the Semantic Latent Space Between Brain and Machine: Similarity is All You Need

Decoding human brain activity with deep learning

Neural encoding with unsupervised spiking convolutional neural network

Explorations of using a convolutional neural network to understand brain activations during movie watching

Modeling 4D fMRI Data via Spatio-Temporal Convolutional Neural Networks (ST-CNN)

Fusing Frontal and Occipital EEG Features to Detect "brain Switch" by Utilizing Convolutional Neural Network.

Unidirectional brain-computer interface: Artificial neural network encoding natural images to fMRI response in the visual cortex

Feature Extraction For Fmri-Based Human Brain Activity Recognition

A Visual Encoding Model Based on Deep Neural Networks and Transfer Learning for Brain Activity Measured by Functional Magnetic Resonance Imaging

Linking Neural Activity to Mental Processes

Investigating the different mechanisms in related neural activities: a focus on auditory perception and imagery

A comparing network for the classification of steady-state visual evoked potential responses based on convolutional neural network

Emergence of Emotion Selectivity in Deep Neural Networks Trained to Recognize Visual Objects

Study of the Human Brain Visual Cortex by Functional Magnetic Resonance Imaging

Neural representations of the perception of handwritten digits and visual objects from a convolutional neural network compared to humans

Decoding Visual fMRI Stimuli from Human Brain Based on Graph Convolutional Neural Network

A Multichannel 2D Convolutional Neural Network Model for Task-Evoked fMRI Data Classification

Novel functional brain network methods based on CNN with an application in proficiency evaluation.