Abstract:The success of current deep saliency models heavily depends on large amounts of annotated human fixation data to fit the highly non-linear mapping between the stimuli and visual saliency. Such fully supervised data-driven approaches are annotation-intensive and often fail to consider the underlying mechanisms of visual attention. In contrast, in this paper, we introduce a model based on various cognitive theories of visual saliency, which learns visual attention patterns in a weakly supervised manner. Our approach incorporates insights from cognitive science as differentiable submodules, resulting in a unified, end-to-end trainable framework. Specifically, our model encapsulates the following important components motivated from biological vision. (a) As scene semantics are closely related to visually attentive regions, our model encodes discriminative spatial information for scene understanding through spatial visual semantics embedding. (b) To model the objectness factors in visual attention deployment, we incorporate object-level semantics embedding and object relation information. (c) Considering the "winner-take-all" mechanism in visual stimuli processing, we model the competition mechanism among objects with softmax based neural attention. (d) Lastly, a conditional center prior is learned to mimic the spatial distribution bias of visual attention. Furthermore, we propose novel loss functions to utilize supervision cues from image-level semantics, saliency prior knowledge, and self-information compression. Experiments show that our method achieves promising results, and even outperforms many of its fully supervised counterparts. Overall, our weakly supervised saliency method makes an essential step towards reducing the annotation budget of current approaches, as well as providing a more comprehensive understanding of the visual attention mechanism. Our code is available at: https://github.com/ashleylqx/WeakFixation.git.

Learning visual saliency based on object's relative relationship

Learning to Model Task-Oriented Attention

Low-level and High-Level Prior Learning for Visual Saliency Estimation.

A Computational Model for Object-Based Visual Saliency: Spreading Attention along Gestalt Cues

A Deep Model of Visual Attention for Saliency Detection on 3D Objects

Image Visual Attention Computation and Application Via the Learning of Object Attributes

Salient Object Detection Via Contrast Information and Object Vision Organization Cues

Combining Background Information and A Top-Down Model for Computing Salient Objects

A Bayesian model for visual salience learning

A Model of Space and Object-based Attention for Visual Saliency

Visual Attention Model Based on Statistical Properties of Neuron Responses.

What Do Deep Saliency Models Learn about Visual Attention?

Visual saliency detection by integrating spatial position prior of object with background cues

Predicting eye fixations with higher-level visual features.

Spatial-Aware Object-Level Saliency Prediction by Learning Graphlet Hierarchies

A Survey of Visual Attention and Saliency Computation

Weakly Supervised Visual Saliency Prediction

New Contour Cue-Based Hybrid Sparse Learning for Salient Object Detection

Object-level Saliency Detection with Color Attributes.

Visual saliency detection based on homology similarity and an experimental evaluation.

A Visual Attention Based Object Detection Model beyond Top-Down and Bottom-up Mechanism