Abstract:Predicting where people look in static scenes, a.k.a visual saliency, has received significant research interest recently. However, relatively less effort has been spent in understanding and modeling visual attention over dynamic scenes. This work makes three contributions to video saliency research. First, we introduce a new benchmark, called DHF1K (Dynamic Human Fixation 1K), for predicting fixations during dynamic scene free-viewing, which is a long-time need in this field. DHF1K consists of 1K high-quality elaborately-selected video sequences annotated by 17 observers using an eye tracker device. The videos span a wide range of scenes, motions, object types and backgrounds. Second, we propose a novel video saliency model, called ACLNet (Attentive CNN-LSTM Network), that augments the CNN-LSTM architecture with a supervised attention mechanism to enable fast end-to-end saliency learning. The attention mechanism explicitly encodes static saliency information, thus allowing LSTM to focus on learning a more flexible temporal saliency representation across successive frames. Such a design fully leverages existing large-scale static fixation datasets, avoids overfitting, and significantly improves training efficiency and testing performance. Third, we perform an extensive evaluation of the state-of-the-art saliency models on three datasets : DHF1K, Hollywood-2, and UCF sports. An attribute-based analysis of previous saliency models and cross-dataset generalization are also presented. Experimental results over more than 1.2K testing videos containing 400K frames demonstrate that ACLNet outperforms other contenders and has a fast processing speed (40 fps using a single GPU). Our code and all the results are available at https://github.com/wenguanwang/DHF1K.

Saliency Prediction with Scene Structural Guidance

A structure-guided approach to the prediction of natural image saliency

Contour-guided saliency detection with long-range interactions

Spatial-Aware Object-Level Saliency Prediction by Learning Graphlet Hierarchies

Revisiting Video Saliency Prediction in the Deep Learning Era

Saliency Detection in Complex Scenes.

Predicting human gaze beyond pixels.

Audiovisual Saliency Prediction Via Deep Learning

Multi-Camera Saliency.

Transcending Pixels: Boosting Saliency Detection via Scene Understanding from Aerial Imagery

Learning to Predict Salient Faces: A Novel Visual-Audio Saliency Model

What Do Deep Saliency Models Learn about Visual Attention?

Saliency In Crowd

Learning To Predict Saliency On Face Images

Attentive and Context-Aware Deep Network for Saliency Prediction on Omni-Directional Images

Deep saliency models learn low-, mid-, and high-level features to predict scene attention

A Computational Model for Stereoscopic Visual Saliency Prediction

Revisiting Video Saliency: A Large-scale Benchmark and a New Model

A Deep Spatial Contextual Long-term Recurrent Convolutional Network for Saliency Detection

Saliency Prediction with External Knowledge

Learning to Model Task-Oriented Attention