Abstract:Detecting saliency in videos is a fundamental step in many computer vision systems. Saliency is the significant target(s) in the video. The object of interest is further analyzed for high-level applications. The segregation of saliency and the background can be made if they exhibit different visual cues. Therefore, saliency detection is often formulated as background subtraction. However, saliency detection is challenging. For instance, dynamic background can result in false positive errors. In another scenario, camouflage will result in false negative errors. With moving cameras, the captured scenes are even more complicated to handle. We propose a new framework, called saliency detection via background model completion (SD-BMC), that comprises a background modeler and a deep learning background/foreground segmentation network. The background modeler generates an initial clean background image from a short image sequence. Based on the idea of video completion, a good background frame can be synthesized with the co-existence of changing background and moving objects. We adopt the background/foreground segmenter, which was pre-trained with a specific video dataset. It can also detect saliency in unseen videos. The background modeler can adjust the background image dynamically when the background/foreground segmenter output deteriorates during processing a long video. To the best of our knowledge, our framework is the first one to adopt video completion for background modeling and saliency detection in videos captured by moving cameras. The F-measure results, obtained from the pan-tilt-zoom (PTZ) videos, show that our proposed framework outperforms some deep learning-based background subtraction models by 11% or more. With more challenging videos, our framework also outperforms many high-ranking background subtraction methods by more than 3%.

Saliency model-based face segmentation and tracking in head-and-shoulder video sequences

Face Segmentation in Head-And-Shoulder Video Sequences Based on Facial Saliency Map

Real-time Detection and Tracking Algorithm Based on the Color and Character of Human Face

Attention Shift-Based Multiple Saliency Object Segmentation.

Accurate Object Segmentation for Video Sequences Via Temporal-Spatial-Frequency Saliency Model.

FaceSeg: automatic face segmentation for real-time video

Saliency Detection with Moving Camera via Background Model Completion

Saliency-Aware Geodesic Video Object Segmentation

Upper Body Human Detection and Segmentation in Low Contrast Video

Video Saliency Detection Using Motion Saliency Filter

Video Saliency Detection via Dynamic Consistent Spatio-Temporal Attention Modelling.

Visual saliency based object tracking

Video Segmentation by Tracking Many Figure-Ground Segments

Video Co-Saliency Guided Co-Segmentation

Visual Feature Extraction Via Eye Tracking for Saliency Driven 2D/3D Registration.

Salient Object Detection Based on Multiscale Segmentation and Fuzzy Broad Learning

Saliency-Aware Video Object Segmentation.

Unsupervized Video Segmentation with Low Depth of Field

Video Saliency Detection Using Object Proposals

Saliency detection using joint spatial-color constraint and multi-scale segmentation

A Segmentation Scheme for Head-Shoulder Video in MPEG Compressed Domain