Abstract:The success of current deep saliency models heavily depends on large amounts of annotated human fixation data to fit the highly non-linear mapping between the stimuli and visual saliency. Such fully supervised data-driven approaches are annotation-intensive and often fail to consider the underlying mechanisms of visual attention. In contrast, in this paper, we introduce a model based on various cognitive theories of visual saliency, which learns visual attention patterns in a weakly supervised manner. Our approach incorporates insights from cognitive science as differentiable submodules, resulting in a unified, end-to-end trainable framework. Specifically, our model encapsulates the following important components motivated from biological vision. (a) As scene semantics are closely related to visually attentive regions, our model encodes discriminative spatial information for scene understanding through spatial visual semantics embedding. (b) To model the objectness factors in visual attention deployment, we incorporate object-level semantics embedding and object relation information. (c) Considering the "winner-take-all" mechanism in visual stimuli processing, we model the competition mechanism among objects with softmax based neural attention. (d) Lastly, a conditional center prior is learned to mimic the spatial distribution bias of visual attention. Furthermore, we propose novel loss functions to utilize supervision cues from image-level semantics, saliency prior knowledge, and self-information compression. Experiments show that our method achieves promising results, and even outperforms many of its fully supervised counterparts. Overall, our weakly supervised saliency method makes an essential step towards reducing the annotation budget of current approaches, as well as providing a more comprehensive understanding of the visual attention mechanism. Our code is available at: https://github.com/ashleylqx/WeakFixation.git.

Saliency-Aware Nonparametric Foreground Annotation Based on Weakly Labeled Data.

Salient Object Detection Using Coarse-to-fine Processing.

Salient Region Detection by Fusing Foreground and Background Cues Extracted from Single Image

Fine-Grained Image Classification Via Spatial Saliency Extraction.

Weakly Supervised Salient Object Detection Using Image Labels

Saliency Guided End-to-end Learning Forweakly Supervised Object Detection

Foreground-Background Collaboration Network for Salient Object Detection.

Saliency Guided End-to-End Learning for Weakly Supervised Object Detection.

Saliency Detection Based on Foreground and Background Extraction

Salient Object Detection via Bounding-box Supervision

Foreground Object Sensing for Saliency Detection

Saliency Detection Via Background and Foreground Seed Selection.

Learning Salient Feature for Salient Object Detection Without Labels

Object-level Saliency Detection with Color Attributes.

Saliency Detection Via Background and Foreground Null Space Learning

Saliency Regularization for Self-Training with Partial Annotations.

Weakly-Supervised Saliency Detection Via Salient Object Subitizing

Saliency detection via foreground rendering and background exclusion

Semi-Supervised Video Salient Object Detection Using Pseudo-Labels

Weakly Supervised Visual Saliency Prediction

Saliency Detection Using Two-Stage Scoring