Abstract:Recovering the background and foreground parts from video frames has important applications in video surveillance. Under the assumption that the background parts are stationary and the foreground are sparse, most of existing methods are based on the framework of robust principal component analysis (RPCA), i.e., modeling the background and foreground parts as a low-rank and sparse matrices, respectively. However, in realistic complex scenarios, the conventional norm sparse regularizer often fails to well characterize the varying sparsity of the foreground components. How to select the sparsity regularizer parameters adaptively according to the local statistics is critical to the success of the RPCA framework for background subtraction task. In this paper, we propose to model the sparse component with a Gaussian scale mixture (GSM) model. Compared with the conventional norm, the GSM-based sparse model has the advantages of jointly estimating the variances of the sparse coefficients (and hence the regularization parameters) and the unknown sparse coefficients, leading to significant estimation accuracy improvements. Moreover, considering that the foreground parts are highly structured, a structured extension of the GSM model is further developed. Specifically, the input frame is divided into many homogeneous regions using superpixel segmentation. By characterizing the set of sparse coefficients in each homogeneous region with the same GSM prior, the local dependencies among the sparse coefficients can be effectively exploited, leading to further improvements for background subtraction. Experimental results on several challenging scenarios show that the proposed method performs much better than most of existing background subtraction methods in terms of both performance and speed.

Foreground Segmentation Via Sparse Representation

Adaptive Foreground and Shadow Segmentation Using Hidden Conditional Random Fields

An Effective Method for Foreground Segmentation of Video

Image Segmentation Using Overlapping Group Sparsity

Foreground Detection Based on Superpixel and Semantic Segmentation

Spatio-Temporal Video Segmentation of Static Scenes and Its Applications

Foreground Segmentation Using Motion Vectors in Sports Video

Robust Bilayer Segmentation and Motion/Depth Estimation with a Handheld Camera

Robust Foreground Segmentation Based on Two Effective Background Models.

Information Integration for Accurate Foreground Segmentation in Complex Scenes

Coarse-to-fine Image Co-segmentation with Intra and Inter Rank Constraints

Spatio-Temporal Segmentation with Depth-Inferred Videos of Static Scenes

Foreground segmentation based on multi-resolution and matting.

Video Segmentation via Multiple Granularity Analysis

Foreground Extraction of Underwater Videos via Sparse and Low-Rank Matrix Decomposition

Video Object Co-Segmentation via Subspace Clustering and Quadratic Pseudo-Boolean Optimization in an MRF Framework

Background initialization and foreground segmentation for bootstrapping video sequences

Automatic foreground extraction in video

Fine-grained Background Representation for Weakly Supervised Semantic Segmentation

An Adaptive Background Modeling Method for Foreground Segmentation

Robust Foreground Estimation via Structured Gaussian Scale Mixture Modeling