Abstract:The success of current deep saliency models heavily depends on large amounts of annotated human fixation data to fit the highly non-linear mapping between the stimuli and visual saliency. Such fully supervised data-driven approaches are annotation-intensive and often fail to consider the underlying mechanisms of visual attention. In contrast, in this paper, we introduce a model based on various cognitive theories of visual saliency, which learns visual attention patterns in a weakly supervised manner. Our approach incorporates insights from cognitive science as differentiable submodules, resulting in a unified, end-to-end trainable framework. Specifically, our model encapsulates the following important components motivated from biological vision. (a) As scene semantics are closely related to visually attentive regions, our model encodes discriminative spatial information for scene understanding through spatial visual semantics embedding. (b) To model the objectness factors in visual attention deployment, we incorporate object-level semantics embedding and object relation information. (c) Considering the "winner-take-all" mechanism in visual stimuli processing, we model the competition mechanism among objects with softmax based neural attention. (d) Lastly, a conditional center prior is learned to mimic the spatial distribution bias of visual attention. Furthermore, we propose novel loss functions to utilize supervision cues from image-level semantics, saliency prior knowledge, and self-information compression. Experiments show that our method achieves promising results, and even outperforms many of its fully supervised counterparts. Overall, our weakly supervised saliency method makes an essential step towards reducing the annotation budget of current approaches, as well as providing a more comprehensive understanding of the visual attention mechanism. Our code is available at: https://github.com/ashleylqx/WeakFixation.git.

Cross-frame feature-saliency mutual reinforcing for weakly supervised video salient object detection

Weakly Supervised Video Salient Object Detection via Point Supervision

CSA-Net: Deep Cross-Complementary Self Attention and Modality-Specific Preservation for Saliency Detection

Weakly-Supervised Salient Object Detection via Scribble Annotations

Video Saliency Prediction Using Enhanced Spatiotemporal Alignment Network

Weakly Supervised RGB-D Salient Object Detection With Prediction Consistency Training and Active Scribble Boosting

A novel seminar learning framework for weakly supervised salient object detection

Co-saliency Detection Via Looking Deep and Wide

A Visual Representation-guided Framework with Global Affinity for Weakly Supervised Salient Object Detection

Complementary characteristics fusion network for weakly supervised salient object detection

Robust Perception and Precise Segmentation for Scribble-Supervised RGB-D Saliency Detection.

DeepSaliency : MultiTask Deep Neural Network Model for Salient Object Detection

RGB-D salient object detection via cross-modal joint feature extraction and low-bound fusion loss

RGB-D Salient Object Detection with Cross-Modality Modulation and Selection

AWANet: Attentive-Aware Wide-Kernels Asymmetrical Network with Blended Contour Information for Salient Object Detection

Progressive Attention-Based Feature Recovery With Scribble Supervision for Saliency Detection in Optical Remote Sensing Image

Saliency-guided video classification via adaptively weighted learning

End-to-End Video Saliency Detection Via a Deep Contextual Spatiotemporal Network

Weakly Supervised Visual Saliency Prediction

Real-time salient object detection with boundary information guidance

Weakly Supervised Video Object Segmentation via Dual-attention Cross-branch Fusion