Abstract:Image sentiment influences visual perception. Emotion-eliciting stimuli such as happy faces and poisonous snakes are generally prioritized in human attention. However, little research has evaluated the interrelationships of image sentiment and visual saliency. In this paper, we present the first study to focus on the relation between emotional properties of an image and visual attention. We first create the EMOtional attention dataset (EMOd). It is a diverse set of emotion-eliciting images, and each image has (1) eye-tracking data collected from 16 subjects, (2) intensive image context labels including object contour object sentiment, object semantic category, and high-level perceptual attributes such as image aesthetics and elicited emotions. We perform extensive analyses on EMOd to identify how image sentiment relates to human attention. We discover an emotion prioritization effect: for our images, emotion-eliciting content attracts human attention strongly, but such advantage diminishes dramatically after initial fixation. Aiming to model the human emotion prioritization computationally, we design a deep neural network for saliency prediction, which includes a novel subnetwork that learns the spatial and semantic context of the image scene. The proposed network outperforms the state-of-the-art on three benchmark datasets, by effectively capturing the relative importance of human attention within an image. The code, models, and dataset are available online at https://nus-sesame.top/emotionalattention/.

A Novel Emotional Saliency Map to Model Emotional Attention Mechanism.

Emotional Attention: A Study of Image Sentiment and Visual Attention

Emotional attention: From eye tracking to computational modeling

A Novel Feature Combination Methods for Saliency-Based Visual Attention

A Novel Method to Study Bottom-up Visual Saliency and its Neural Mechanism

A Biologically Inspired Model of Emotional Attention

A Novel Approach for Visual Saliency Detection and Segmentation Based on Objectness and Top-Down Attention

A Novel Visual Saliency Analysis Model Based on Dynamic Multiple Feature Combination Strategy

A Visual-Attention Model Using Earth Mover's Distance-Based Saliency Measurement and Nonlinear Feature Combination

A Biological Inspired Features Based Saliency Map

A Novel Attention Based CNN Model for Emotion Intensity Prediction.

Visual Saliency Maps Can Apply to Facial Expression Recognition

What Do Deep Saliency Models Learn about Visual Attention?

Video Saliency Detection via Dynamic Consistent Spatio-Temporal Attention Modelling.

A biologically inspired computational model for image saliency detection.

Saliency Detection by Non-Linear Intensity Mapping in Images

Adaptive Non-Linear Intensity Mapping Based Salient Region Extraction

A Computational Model for Object-Based Visual Saliency: Spreading Attention along Gestalt Cues

Analysis and realization of saliency map based on visual attention mechanism

Visual Attention Model Based on Statistical Properties of Neuron Responses.

A VISUAL ATTENTION MODEL FOR NATURAL SCENES BASED ON DYNAMIC FEATURE COMBINATION