Abstract:This paper presents a real-time spatiotemporal segmentation approach to extract video objects in the H.264 compressed domain. The only exploited segmentation cue is the motion vector (MV) field extracted from the H.264 compressed video. MV field is first temporally and spatially normalized and then accumulated by an iteratively backward projection scheme to enhance the salient motion. Then global motion compensation is performed on the accumulated MV field, which is also moderately segmented into different motion-homogenous regions by a modified statistical region growing algorithm. The hypothesis testing using the block residuals of global motion compensation is employed for intra-frame classification of segmented regions, and the projection is exploited for inter-frame tracking of previous video objects. Using the above results of intra-frame classification and inter-frame tracking as input, a correspondence matrix based spatiotemporal segmentation approach is proposed to segment video objects under different situations including appearing and disappearing objects, splitting and merging objects, stopping moving objects, multiple object tracking and scene change in a unified and efficient way. Experimental results for several H.264 compressed video sequences demonstrate the real-time performance and good segmentation quality of the proposed approach.

An Efficient Automatic Video Segmentation Method Based on Intersection of Frame Differences

An algorithm for automatic segmentation of video objects based on intra-frame image partition

Video Shot Segmentation Method Using Linear Segmentation and Sequences Combination

Detection and Segmentation of Moving Objects Using Temporal and Spatial Cues

Video Stream Segmentation Method Based on Video Page

Improved Intelligent Scissors and Snake based VOP Interpolation for Semiautomatic Video Object Segmentation

An Adaptive Frame Skipping and Vop Interpolation Algorithm for Video Object Segmentation

An object segmentation method for the color slow-motion videos based on adjacent frames gradual change

Spatio-Temporal Video Segmentation of Static Scenes and Its Applications

Multiresolution Segmentation of Video Objects in the Compression Domain

Object Segmentation Using Graph Cuts for the H.264 Compressed Video with Moving Background

Real-time spatiotemporal segmentation of video objects in the H.264 compressed domain

Contour Based Automatic Scene Segmentation in Image Sequences

Automatic Segmentation of Moving Objects in Video Sequences Based on Dynamic Background Construction

Semi-automatic Video Object Segmentation Using Seeded Region Merging and Bidirectional Projection

An automatic segmentation algorithm for moving objects in video sequences under multi-constraints

Automatic Segmentation of Moving Objects in Video Sequences Based on Spatio-Temporal Information

Efficient video object segmentation based on Gaussian mixture model and Markov random field

Object segmentation based on disparity estimation

A Novel Framework for Semi-Automatic Video Object Segmentation

Supervised Video Object Segmentation Using a Small Number of Interactions.