Abstract:Images contain rich information and can induce various emotions in the audience. Image emotion classification aims to identify the emotion categories that images can evoke. It is widely used in mental health assessment, human–computer interaction, etc. There are two main problems in existing image emotion classification methods: (1) Most of them only focus on a single emotion label; (2) The global structural relationship among semantic objects in the image is ignored. Therefore, this paper proposes an Image Emotion classification method based on Multi-Graph Multi-Label learning (IE-MGML). In contrast to the existing approaches, the image is transformed into a graph-based representation by extracting the emotional features of semantic objects and calculating the similarity between the features. The local (semantic objects) features and global structure (relationship among semantic objects) features of the image are fused by the relationship between nodes. Furthermore, the graph representation of an image from the perspective of multiple emotional features is pooled and modeled as a graph bag containing multiple graphs (i.e., multi-graph). In multi-graph learning, the graph kernel directly evaluates a graph-label dependency score to avoid the loss of structural information caused by graph-instance degradation. The bag(image)-label dependency score is obtained by aggregating the graph-label dependency score from different perspectives through the aggregation function. The problem of error accumulation in the learning process is handled by proposing a threshold-based ranking loss objective function. Moreover, the non-convex optimization problem is addressed using a subgradient descent algorithm to deal with the required high-dimensional space computation. Experimental results on three general image emotion datasets show that the proposed method outperforms the state-of-the-art methods.

Learning Multi-level Deep Representations for Image Emotion Classification

Learning Multi-level Representations for Image Emotion Recognition in the Deep Convolutional Network

Learning multi-level representations for affective image recognition

Image Emotion Recognition Based on Deep Neural Network

Dependency Exploitation: A Unified CNN-RNN Approach for Visual Emotion Recognition

A New Deep Learning Method for Multi-label Facial Expression Recognition Based on Local Constraint Features

Multiscale Emotion Representation Learning for Affective Image Recognition

A supervised contrastive learning-based model for image emotion classification

Multi-Output Learning Based on Multimodal GCN and Co-Attention for Image Aesthetics and Emotion Analysis

Exploring Discriminative Representations for Image Emotion Recognition with CNNs.

Multi-Feature Fusion Based Deep Network for Image Semantic Recognition

Multimodal Emotion Classification Method Based on Multilevel Deep Convolution Neural Network in Social Networks

A Multi-feature Fusion and SSAE-Based Deep Network for Image Semantic Recognition

Speech Emotion Recognition Via Multi-Level Attention Network

Human Emotion Recognition with Electroencephalographic Multidimensional Features by Hybrid Deep Neural Networks

Multi-modal Facial Expression Feature Based on Deep-Neural Networks

Multimodal Emotion Classification with Multi-Level Semantic Reasoning Network

Image Aesthetic Assessment Based on Emotion-Assisted Multi-Task Learning Network

A Multi-Stage Visual Perception Approach for Image Emotion Analysis

A Novel and Powerful Dual-Stream Multi-Level Graph Convolution Network for Emotion Recognition

Image Emotion Multi-Label Classification Based on Multi-Graph Learning