Abstract:Static and moving objects often occur in real-life videos. Most video object segmentation methods only focus on extracting and exploiting motion cues to perceive moving objects. Once faced with the frames of static objects, the moving object predictors may predict failed results caused by uncertain motion information, such as low-quality optical flow maps. Besides, different sources such as RGB, depth, optical flow and static saliency can provide useful information about the objects. However, existing approaches only consider either the RGB or RGB and optical flow. In this paper, we propose a novel adaptive multi-source predictor for zero-shot video object segmentation (ZVOS). In the static object predictor, the RGB source is converted to depth and static saliency sources, simultaneously. In the moving object predictor, we propose the multi-source fusion structure. First, the spatial importance of each source is highlighted with the help of the interoceptive spatial attention module (ISAM). Second, the motion-enhanced module (MEM) is designed to generate pure foreground motion attention for improving the representation of static and moving features in the decoder. Furthermore, we design a feature purification module (FPM) to filter the inter-source incompatible features. By using the ISAM, MEM and FPM, the multi-source features are effectively fused. In addition, we put forward an adaptive predictor fusion network (APF) to evaluate the quality of the optical flow map and fuse the predictions from the static object predictor and the moving object predictor in order to prevent over-reliance on the failed results caused by low-quality optical flow maps. Experiments show that the proposed model outperforms the state-of-the-art methods on three challenging ZVOS benchmarks. And, the static object predictor precisely predicts a high-quality depth map and static saliency map at the same time.

Object-oriented Video Prediction with Pixel-Level Attention.

Human Visual Perception Based Image Quality Assessment for Video Prediction

Temporal Consistent Object Pose Estimation from Monocular Videos

Object-centric Video Prediction without Annotation

Adaptive Hierarchical Motion-Focused Model for Video Prediction.

Pixel Objectness: Learning to Segment Generic Objects Automatically in Images and Videos

Adaptive Multi-source Predictor for Zero-shot Video Object Segmentation

Learning Physical Dynamics for Object-centric Visual Prediction

STAU: A SpatioTemporal-Aware Unit for Video Prediction and Beyond

Pair-wise Layer Attention with Spatial Masking for Video Prediction

Objects do not disappear: Video object detection by single-frame object location anticipation

Object-Centric Video Prediction via Decoupling of Object Dynamics and Interactions

From Single to Multiple: Leveraging Multi-level Prediction Spaces for Video Forecasting

Predicting Video Saliency with Object-to-Motion CNN and Two-layer Convolutional LSTM

Looking-Ahead: Neural Future Video Frame Prediction

Revisiting Hierarchical Approach for Persistent Long-Term Video Prediction

Spatial and Temporal Visual Attention Prediction in Videos Using Eye Movement Data

Predicting Long-horizon Futures by Conditioning on Geometry and Time

Video Frame Prediction by Deep Multi-Branch Mask Network

Weakly Supervised Video Salient Object Detection via Point Supervision