Abstract:Images play a vital role in social media platforms, which can more vividly reflect people's inner emotions and preferences, so visual sentiment analysis has become an important research topic. In this paper, we propose a Supervised Contrastive Learning-based model for image emotion classification, which consists of two modules of low-level feature extraction and deep emotional feature extraction, and feature fusion is used to enhance the overall perception of image emotions. In the low-level feature extraction module, the LBP-U (Local Binary Patterns with Uniform Patterns) algorithm is employed to extract texture features from the images, which can effectively capture the texture information of the images, aiding in the differentiation of images belonging to different emotion categories. In the deep emotional feature extraction module, we introduce a Supervised Contrastive Learning approach to improve the extraction of deep emotional features by narrowing the intra-class distance among images of the same emotion category while expanding the inter-class distance between images of different emotion categories. Through fusing the low-level and deep emotional features, our model comprehensively utilizes features at different levels, thereby enhancing the overall emotion classification performance. To assess the classification performance and generalization capability of the proposed model, we conduct experiments on the publicly FI (Flickr and Instagram) Emotion dataset. Comparative analysis of the experimental results demonstrates that our proposed model has good performance for image emotion classification. Additionally, we conduct ablation experiments to analyze the impact of different levels of features and various loss functions on the model's performance, thereby validating the superiority of our proposed approach.

Joint Image Emotion Classification and Distribution Learning Via Deep Convolutional Neural Network.

Image Emotion Distribution Learning With Graph Convolutional Networks

Text Emotion Distribution Learning Via Multi-Task Convolutional Neural Network

Learning Visual Sentiment Distributions via Augmented Conditional Probability Neural Network

Convolutional Neural Network Classification loss MultiTask Loss Similarity Loss y Feature Embedding Sentiment Constraint

Retrieving and Classifying Affective Images Via Deep Metric Learning

Image Emotion Distribution Learning Based on Enhanced Fuzzy KNN Algorithm with Sparse Learning

Robust Local-Global Feature Representation for Image Emotion Distribution Learning

Learning Visual Emotion Distributions via Multi-Modal Features Fusion.

Discovering Affective Regions in Deep Convolutional Neural Networks for Visual Sentiment Prediction.

Lexicon-Enhanced Multi-Task Convolutional Neural Network for Emotion Distribution Learning

A Saliency Detection and Gram Matrix Transform-Based Convolutional Neural Network for Image Emotion Classification.

Predicting Image Emotion Distribution by Learning Labels' Correlation.

Joint Binary Neural Network for Multi-label Learning with Applications to Emotion Classification

Learning Multi-level Representations for Image Emotion Recognition in the Deep Convolutional Network

Joint Weakly Supervised Image Emotion Analysis Based on Interclass Discrimination and Intraclass Correlation

Adaptive Deep Metric Learning for Affective Image Retrieval and Classification

Correction To: Deep Transfer Learning for Image Emotion Analysis: Reducing Marginal and Joint Distribution Discrepancies Together

Lexicon-based Conversion Strategy Words Sentence Single Label Joy : 0 Fear : 0 Anger : 0 Surprise : 0 Sadness : 1

Label Distribution Amendment with Emotional Semantic Correlations for Facial Expression Recognition

A supervised contrastive learning-based model for image emotion classification