Abstract:Previous studies have demonstrated that images are of great importance in attracting people’s attention and motivating them to take action. Various attributes (e.g., colors, aesthetics, and embedded objects) related to images are considered driving factors. Among which emotions in images, in particular, play a critical role in stimulating individuals, based on the Stimulus–Organism–Response theory. Consequently, many researchers put great efforts to understand image emotions, ranging from developing theoretical models to a broad spectrum of applications. Due to the complex and unstructured characteristics of images, identifying image emotions is challenging. Although some significant progress in image emotion classification has been achieved, inherent constraints still remain unaddressed. For example, acquiring a sufficiently large amount of labeled data to train a good model is costly and inevitably requires lots of human efforts. Besides, building a generalized model applicable to different datasets still needs a deep exploration. Image emotions are very subjective, which also makes such a classification task difficult. This paper proposes a general meta-learning framework for the few-shot image emotion classification, called Meta-IEC. Meta-IEC provides the capability of: (i) adapting to a similar dataset but new classes that have not been encountered before, and (ii) generalizing to a completely different dataset where emotion classes are unseen in the training dataset and only very few labeled images are available. Meta-IEC is also able to capture the uncertainty and ambiguity during the meta-testing, where we implement a hierarchical Bayesian graphical model to understand latent relationships among various parameters between meta-training and meta-testing. Extensive experiments on three commonly used datasets empirically demonstrate the superiority of our method over several state-of-the-art baselines. For example, our meta-learning based model can achieve performance improvement up to 5+%. We also provide some managerial implications on parameter sensitivity and label selection of meta-training and meta-testing.

Learning User-Emotion and User-Feature Couplings for Image Emotion Classification

Doubled Coupling for Image Emotion Distribution Learning.

Exploring Feature Coupling and Model Coupling for Image Source Identification

Exploiting Emotional Concepts for Image Emotion Recognition

One for All: A Unified Generative Framework for Image Emotion Classification

Image Emotion Computing

Emotion Recognition via Environmental Context and Human Body

Joint Weakly Supervised Image Emotion Analysis Based on Interclass Discrimination and Intraclass Correlation

Learning Meta-Knowledge for Few-Shot Image Emotion Recognition

Learning Visual Emotion Distributions via Multi-Modal Features Fusion.

Image Emotion Analysis Based on Semantic Concepts

Emotion-wise feature interaction analysis-based visual emotion distribution learning

A supervised contrastive learning-based model for image emotion classification

Affective Image Classification Using Multi-Scale Emotion Factorization Features

Image Emotion Distribution Learning Based on Enhanced Fuzzy KNN Algorithm with Sparse Learning

Learning Multi-level Representations for Image Emotion Recognition in the Deep Convolutional Network

Computational Emotion Analysis From Images: Recent Advances and Future Directions

Modality Coupling for Privacy Image Classification

Multi-Output Learning Based on Multimodal GCN and Co-Attention for Image Aesthetics and Emotion Analysis

Modeling Emotion Influence from Images in Social Networks

Adaptive Deep Metric Learning for Affective Image Retrieval and Classification