Abstract:Semantics and motion are two cues of essence for the success in video salient object detection. Most existing deep-learning based approaches extract semantic features by the use of only one fully convolutional network with simple stacked encoders. They simulate motion patterns of video objects with two consecutive frames being simultaneously fed into a convolutional LSTM network or a weights-sharing fully convolutional network. However, such approaches have the shortcomings of producing a coarse predicted saliency map or requiring significant computational overheads. In this paper, we present a novel approach with cascaded fully convolutional networks involving motion attention (abbreviated as CFCN-MA), to achieve real-time saliency detection in videos. Our key idea is to construct twofold fully convolutional networks in order to gain a saliency map from coarse to fine. We devise an optical flow-based motion attention mechanism to improve the prediction accuracy of the initial fully convolutional networks, using the popular FlowNet2-SD model that is efficient and effective for motion pattern recognition of distinctive objects in videos. This method can obtain a fine saliency map with a refined region of interest. Moreover, we propose a means for calculating attention-guided intersection-over-union loss (shortnamed as AIoU) to supervise the CFCN-MA model in learning a saliency map with both clear edge and complete structure. Our approach is evaluated on three popular benchmark datasets, namely DAVIS, ViSal and FBMS. Experimental results demonstrate that our method outperforms many state-of-the-art techniques while meeting the real-time demand at 27 fps.

Video attention saliency mapping using pulse coupled neural network and optical flow

Moving Object Attention Selection Using Optical Flow And Pulse Coupled Neural Network

Moving Target Tracking Based on Pulse Coupled Neural Network and Optical Flow.

Video Saliency Detection via Dynamic Consistent Spatio-Temporal Attention Modelling.

Integrating Bottom-Up and Top-Down Visual Stimulus for Saliency Detection in News Video

A visual attention model for news video

Saliency Flow Based Video Segmentation Via Motion Guided Contour Refinement.

Pulse Coupled Neural Network Based Topological Properties Applied in Attention Saliency Detection.

Human Vision Attention Mechanism-Inspired Temporal-Spatial Feature Pyramid for Video Saliency Detection

Video Saliency Prediction With Optimized Optical Flow And Gravity Center Bias

TSMSAN: A Three-Stream Multi-Scale Attentive Network for Video Saliency Detection.

Motion Guided Attention for Video Salient Object Detection.

Video Saliency Detection Based on Eye-Movement Guided Region Matching and LDP Embedded Optical Flow

Hybrid Convolutional Neural Networks and Optical Flow for Video Visual Attention Prediction.

Color and motion information fusion based saliency filter for video

Progressively Real-Time Video Salient Object Detection Via Cascaded Fully Convolutional Networks with Motion Attention.

Attention Embedded Spatio-Temporal Network for Video Salient Object Detection

End-to-end Visual Object Tracking with Motion Saliency Guidance

A Video Salient Object Detection Model Guided By Spatio-Temporal Prior

A visual attention model for video based on non- Negative matrix factorization sparseness on parts

Video Saliency Object Detection with Motion Quality Compensation