Abstract:Extracting spatio-temporally consistent segments from a video sequence is a challenging problem due to the complexity of color, motion and occlusions. Most existing spatio-temporal segmentation approaches rely on pairwise motion estimation, which have inherent difficulties in handling large displacement with significant occlusions. This paper presents a novel spatio-temporal segmentation method for depth-inferred videos. The depth data of input videos can be estimated beforehand using a multiview stereo technique. Our framework consists of two steps. In the first step, in order to make the extracted 2D segments temporally consistent, we introduce a spatial segmentation based on the probabilistic boundary maps, by collecting the boundary statistics in a video. In the second step, the consistent 2D segments in different frames are matched to initialize the volume segments. Then we compute the segment probability for each pixel by projecting it to other frames to collect the statistics, and incorporate it into the spatio-temporal segmentation energy to explicitly enforce temporal coherence constraint. The spatio-temporal segmentation results are iteratively refined in a video, so that a set of spatio-temporally consistent volume segments are finally achieved. The effectiveness of our automatic method is demonstrated using a variety of challenging video examples.

New Real-Time Algorithm for Multiview Disparity-Based 3D-Object Segmentation.

Object segmentation based on disparity estimation

Space-temporal stereo matching for dynamic scene

A New Approach of Point Cloud Processing and Scene Segmentation for Guiding the Visually Impaired

Object segmentation using stereo images

A Stereo Video Object Segmentation Algorithm Based on Motion Detection and Disparity

Spatio-Temporal Video Segmentation of Static Scenes and Its Applications

Object segmentation based on accurate disparity estimation

Accurate Real-Time Stereo Correspondence Using Intra- and Inter-Scanline Optimization

Disparity Estimation and Virtual View Synthesis from Stereo Video

Segmentation-based dense stereo matching algorithm for virtual view synthesis

Semi-Automatic 2D-to-3D Conversion Using Disparity Propagation

Intermediate View Synthesis and Interpolation Based on Disparity Calibration and Region Segmentation

Fast Automatic Elimination of Vertical Parallax of Multiview Images

An Adaptive Frame Skipping and Vop Interpolation Algorithm for Video Object Segmentation

A multiview video quality assessment method based on disparity and SSIM

Real-time spatiotemporal segmentation of video objects in the H.264 compressed domain

Spatio-Temporal Segmentation with Depth-Inferred Videos of Static Scenes

A Temporally Streamlined Optimization Method for Stereo Video Correspondence

Real-time Stereo to Multi-View Conversion System Based on Adaptive Meshing

Segment-Based Disparity Refinement With Occlusion Handling for Stereo Matching