Abstract:Conventional video saliency detection methods frequently follow the common bottom-up thread to estimate video saliency within the short-term fashion. As a result, such methods can not avoid the obstinate accumulation of errors when the collected low-level clues are constantly ill-detected. Also, being noticed that a portion of video frames, which are not nearby the current video frame over the time axis, may potentially benefit the saliency detection in the current video frame. Thus, we propose to solve the aforementioned problem using our newly-designed key frame strategy (KFS), whose core rationale is to utilize both the spatial-temporal coherency of the salient foregrounds and the objectness prior (i.e., how likely it is for an object proposal to contain an object of any class) to reveal the valuable long-term information. We could utilize all this newly-revealed long-term information to guide our subsequent "self-paced" saliency diffusion, which enables each key frame itself to determine its diffusion range and diffusion strength to correct those ill-detected video frames. At the algorithmic level, we first divide a video sequence into short-term frame batches, and the object proposals are obtained in a frame-wise manner. Then, for each object proposal, we utilize a pre-trained deep saliency model to obtain high-dimensional features in order to represent the spatial contrast. Since the contrast computation within multiple neighbored video frames (i.e., the non-local manner) is relatively insensitive to the appearance variation, those object proposals with high-quality low-level saliency estimation frequently exhibit strong similarity over the temporal scale. Next, the long-term common consistency (e.g., appearance models/movement patterns) of the salient foregrounds could be explicitly revealed via similarity analysis accordingly. We further boost the detection accuracy via long-term information guided saliency diffusion in a self-paced manner. We have conducted extensive experiments to compare our method with 16 state-of-the-art methods over 4 largest public available benchmarks, and all results demonstrate the superiority of our method in terms of both accuracy and robustness.

Saliency Detection Via Coarse-to-fine Diffusion-Based Compactness with Weighted Learning Affinity Matrix

Salient Object Detection Via Cross Diffusion-Based Compactness on Multiple Graphs

Graph-based Saliency Detection Using a Learning Joint Affinity Matrix

Intensifying Graph Diffusion-Based Salient Object Detection with Sparse Graph Weighting

Graph Construction by Incorporating Local and Global Affinity Graphs for Saliency Detection

Saliency Detection by Compactness Diffusion.

Multiview Diffusion-Based Affinity Graph Learning with Good Neighbourhoods for Salient Object Detection

Robust manifold-preserving diffusion-based saliency detection by adaptive weight construction.

Super Diffusion for Salient Object Detection.

Learning Full-Range Affinity for Diffusion-Based Saliency Detection.

Salient Region Detection by Fusing Foreground and Background Cues Extracted from Single Image

Salient Region Detection via Integrating Diffusion-Based Compactness and Local Contrast

Salient Object Detection Using Coarse-to-fine Processing.

Influence Updating and Compactness Enhanced Approach for Saliency Detection

Salient Object Detection Based on Compactness and Foreground Connectivity

A joint similarity matrix learning of multi-view data for RGB-T saliency detection

Video Saliency Detection Via Spatial-Temporal Fusion and Low-Rank Coherency Diffusion

Exploiting Background Divergence and Foreground Compactness for Salient Object Detection

Saliency Detection by Hierarchically Integrating Compactness, Contrast and Boundary Connectivity.

Accurate and Robust Video Saliency Detection Via Self-Paced Diffusion.

Self-Adaptively Weighted Co-Saliency Detection Via Rank Constraint