Abstract:Visual Emotion Analysis (VEA), which aims to predict people's emotions towards different visual stimuli, has become an attractive research topic recently. Rather than a single label classification task, it is more rational to regard VEA as a Label Distribution Learning (LDL) problem by voting from different individuals. Existing methods often predict visual emotion distribution in a unified network, neglecting the inherent subjectivity in its crowd voting process. In psychology, the \textit{Object-Appraisal-Emotion} model has demonstrated that each individual's emotion is affected by his/her subjective appraisal, which is further formed by the affective memory. Inspired by this, we propose a novel \textit{Subjectivity Appraise-and-Match Network (SAMNet)} to investigate the subjectivity in visual emotion distribution. To depict the diversity in crowd voting process, we first propose the \textit{Subjectivity Appraising} with multiple branches, where each branch simulates the emotion evocation process of a specific individual. Specifically, we construct the affective memory with an attention-based mechanism to preserve each individual's unique emotional experience. A subjectivity loss is further proposed to guarantee the divergence between different individuals. Moreover, we propose the \textit{Subjectivity Matching} with a matching loss, aiming at assigning unordered emotion labels to ordered individual predictions in a one-to-one correspondence with the Hungarian algorithm. Extensive experiments and comparisons are conducted on public visual emotion distribution datasets, and the results demonstrate that the proposed SAMNet consistently outperforms the state-of-the-art methods. Ablation study verifies the effectiveness of our method and visualization proves its interpretability.

Learning Image Aesthetic Subjectivity from Attribute-Aware Relational Reasoning Network

Context-aware Attention Network for Predicting Image Aesthetic Subjectivity

Personalized Image Aesthetics Assessment Via Multi-Attribute Interactive Reasoning

A Comprehensive Survey on Computational Aesthetic Evaluation of Visual Art Images: Metrics and Challenges

Distribution-Oriented Aesthetics Assessment with Semantic-Aware Hybrid Network

Learning Subjective Attributes of Images from Auxiliary Sources

Collaborative and Attentive Learning for Personalized Image Aesthetic Assessment

Image Aesthetic Assessment Assisted by Attributes Through Adversarial Learning.

Photo Aesthetics Ranking Network with Attributes and Content Adaptation

Learning Personalized Image Aesthetics from Subjective and Objective Attributes

Considering User Agreement in Learning to Predict the Aesthetic Quality

Learning Image Aesthetic Assessment from Object-level Visual Components

Multiple Aesthetic Attribute Assessment By Exploiting Relations Among Aesthetic Attributes

Aesthetic Attribute Assessment of Images Numerically on Mixed Multi-attribute Datasets

Image Aesthetics Assessment With Emotion-Aware Multibranch Network

Learning Relative Aesthetic Quality with a Pairwise Approach.

Seeking Subjectivity in Visual Emotion Distribution Learning

Image Aesthetics Assessment Based on User Social Behavior.

Query-Dependent Aesthetic Model with Deep Learning for Photo Quality Assessment

A Computational Approach to Relative Aesthetics

Spatial Attentive Image Aesthetic Assessment