Abstract:It is a challenging task to extract segmentation mask of a target from a single noisy video, which involves object discovery coupled with segmentation. To solve this challenge, we present a method to jointly discover and segment an object from a noisy video, where the target disappears intermittently throughout the video. Previous methods either only fulfill video object discovery, or video object segmentation presuming the existence of the object in each frame. We argue that jointly conducting the two tasks in a unified way will be beneficial. In other words, video object discovery and video object segmentation tasks can facilitate each other. To validate this hypothesis, we propose a principled probabilistic model, where two dynamic Markov networks are coupled-one for discovery and the other for segmentation. When conducting the Bayesian inference on this model using belief propagation, the bi-directional message passing reveals a clear collaboration between these two inference tasks. We validated our proposed method in five data sets. The first three video data sets, i.e., the SegTrack data set, the YouTube-objects data set, and the Davis data set, are not noisy, where all video frames contain the objects. The two noisy data sets, i.e., the XJTU-Stevens data set, and the Noisy-ViDiSeg data set, newly introduced in this paper, both have many frames that do not contain the objects. When compared with state of the art, it is shown that although our method produces inferior results on video data sets without noisy frames, we are able to obtain better results on video data sets with noisy frames.

Video Object Segmentation Research Based on Features Joint Modeling

Adaptive Foreground and Shadow Segmentation Using Hidden Conditional Random Fields

Efficient video object segmentation based on Gaussian mixture model and Markov random field

Human object segmentation using Gaussian mixture model and graph cuts

Video object segmentation by motion-based sequential feature clustering.

Object segmentation based on disparity estimation

Video object segmentation via couple streams and feature memory

Non-rigid Video Object Segmentation Based on Semantic Multi-level Framework.

Collaborative Video Object Segmentation by Foreground-Background Integration

Object Segmentation Using Graph Cuts for the H.264 Compressed Video with Moving Background

Joint Attention Mechanism for Unsupervised Video Object Segmentation

Semi-automatic Video Object Segmentation Using Seeded Region Merging and Bidirectional Projection

Video Object Segmentation by Integrating Trajectories from Points and Regions

Robust Video Object Cosegmentation.

Video Object Co-Segmentation via Subspace Clustering and Quadratic Pseudo-Boolean Optimization in an MRF Framework

Joint Modeling of Feature, Correspondence, and a Compressed Memory for Video Object Segmentation

An Effective Method for Foreground Segmentation of Video

Joint Video Object Discovery and Segmentation by Coupled Dynamic Markov Networks

Video Segmentation Using Multiple Features Based on EM Algorithm

Video Foreground Segmentation Based on Sequential Feature Clustering

Co-Training Based Segmentation of Merged Moving Objects