Abstract:The success of current deep saliency models heavily depends on large amounts of annotated human fixation data to fit the highly non-linear mapping between the stimuli and visual saliency. Such fully supervised data-driven approaches are annotation-intensive and often fail to consider the underlying mechanisms of visual attention. In contrast, in this paper, we introduce a model based on various cognitive theories of visual saliency, which learns visual attention patterns in a weakly supervised manner. Our approach incorporates insights from cognitive science as differentiable submodules, resulting in a unified, end-to-end trainable framework. Specifically, our model encapsulates the following important components motivated from biological vision. (a) As scene semantics are closely related to visually attentive regions, our model encodes discriminative spatial information for scene understanding through spatial visual semantics embedding. (b) To model the objectness factors in visual attention deployment, we incorporate object-level semantics embedding and object relation information. (c) Considering the "winner-take-all" mechanism in visual stimuli processing, we model the competition mechanism among objects with softmax based neural attention. (d) Lastly, a conditional center prior is learned to mimic the spatial distribution bias of visual attention. Furthermore, we propose novel loss functions to utilize supervision cues from image-level semantics, saliency prior knowledge, and self-information compression. Experiments show that our method achieves promising results, and even outperforms many of its fully supervised counterparts. Overall, our weakly supervised saliency method makes an essential step towards reducing the annotation budget of current approaches, as well as providing a more comprehensive understanding of the visual attention mechanism. Our code is available at: https://github.com/ashleylqx/WeakFixation.git.

Improving Children'S Gaze Prediction Via Separate Facial Areas And Attention Shift Cue

Learning Stereoscopic Visual Attention Model for 3d Video

Learning to Model Task-Oriented Attention

Learning to Predict Salient Faces: A Novel Visual-Audio Saliency Model

FMRI-Guided Time-Symmetric Joint Model for Visual Attention Prediction.

Visual Attention Analysis and Prediction on Human Faces for Children with Autism Spectrum Disorder

An Attention Detection System Based on Gaze Estimation Using Self-supervised Learning

Predicting eye fixations with higher-level visual features.

Specialized Gaze Estimation for Children by Convolutional Neural Network and Domain Adaptation.

LEARNING TO PREDICT WHERE THE CHILDREN WITH ASD LOOK

Visual Attention on Human Face

Visual Attention Modeling Based on Short-Term Environmental Adaption

Spatial and Temporal Visual Attention Prediction in Videos Using Eye Movement Data

An Attention Selection Model With Visual Memory And Online Learning

Predicting human gaze beyond pixels.

Weakly Supervised Visual Saliency Prediction

Computational Attention System for Children, Adults and Elderly

Fixation prediction through multimodal analysis

Gaze Estimation in Children's Peer-Play Scenarios

Toward Statistical Modeling of Saccadic Eye-Movement and Visual Saliency.

A Reverse Hierarchy Model for Predicting Eye Fixations