Abstract:With the development of social networks and intelligent terminals, it is becoming more convenient to share and acquire images. The massive growth of the number of social images makes people have higher demands for automatic image processing, especially in the aesthetic and emotional perspective. Both aesthetics assessment and emotion recognition require a higher ability for the computer to simulate high-level visual perception understanding, which belongs to the field of image processing and pattern recognition. However, existing methods often ignore the prior knowledge of images and intrinsic relationships between aesthetic and emotional perspectives. Recently, machine learning and deep learning have become powerful methods for researchers to solve mathematical problems in computing, such as image processing and pattern recognition. Both images and abstract concepts can be converted into numerical matrices and then establish the mapping relations using mathematics on computers. In this work, we propose an end-to-end multi-output deep learning model based on multimodal Graph Convolutional Network (GCN) and co-attention for aesthetic and emotion conjoint analysis. In our model, a stacked multimodal GCN network is proposed to encode the features under the guidance of the correlation matrix, and a co-attention module is designed to help the aesthetics and emotion feature representation learn from each other interactively. Experimental results indicate that our proposed model achieves competitive performance on the IAE dataset. Progressive results on the AVA and ArtPhoto datasets also prove the generalization ability of our model.

Image Emotion Distribution Learning With Graph Convolutional Networks

G-GCSN: Global Graph Convolution Shrinkage Network for Emotion Perception from Gait

Multi-Output Learning Based on Multimodal GCN and Co-Attention for Image Aesthetics and Emotion Analysis

Graph Convolutional Network with Connectivity Uncertainty for EEG-based Emotion Recognition

Emotional Attention Detection and Correlation Exploration for Image Emotion Distribution Learning.

Predicting Continuous Probability Distribution Of Image Emotions In Valence-Arousal Space

A Multi-Dimensional Graph Convolution Network for EEG Emotion Recognition

EEG-Based Emotion Recognition Using Trainable Adjacency Relation Driven Graph Convolutional Network

EmoGraph: Capturing Emotion Correlations using Graph Networks

Label Distribution Amendment with Emotional Semantic Correlations for Facial Expression Recognition

Learning Visual Emotion Distributions via Multi-Modal Features Fusion.

Predicting Personalized Image Emotion Perceptions in Social Networks

StyleEDL: Style-Guided High-order Attention Network for Image Emotion Distribution Learning

A Novel and Powerful Dual-Stream Multi-Level Graph Convolution Network for Emotion Recognition

Emotion-wise feature interaction analysis-based visual emotion distribution learning

Speech Emotion Recognition Based on Temporal-Spatial Learnable Graph Convolutional Neural Network

EmotionGAN: Unsupervised Domain Adaptation for Learning Discrete Probability Distributions of Image Emotions.

Discrete Probability Distribution Prediction of Image Emotions with Shared Sparse Learning

Continuous Probability Distribution Prediction of Image Emotions Via Multitask Shared Sparse Regression

Image Emotion Computing

EmoGen: Emotional Image Content Generation with Text-to-Image Diffusion Models