Abstract:The foreground segmentation method is a crucial first step for many video analysis methods such as action recognition and object tracking. In the past five years, convolutional neural network based foreground segmentation methods have made a great breakthrough. However, most of them pay more attention to stationary cameras and have constrained performance on the pan-tilt-zoom (PTZ) cameras. In this paper, an end-to-end deep features homography transformation and fusion network based foreground segmentation method (HTFnetSeg) is proposed for surveillance videos recorded by PTZ cameras. In the kernel of HTFnetSeg, there is the combination of an unsupervised semantic attention homography estimation network (SAHnet) for frames alignment and a spatial transformed deep features fusion network (STDFFnet) for segmentation. The semantic attention mask in SAHnet reinforces the network to focus on background alignment by reducing the noise that comes from the foreground. STDFFnet is designed to reuse the deep features extracted during the semantic attention mask generation step by aligning the features rather than only the frames, with a spatial transformation technique in order to reduce the algorithm complexity. Additionally, a conservative strategy is proposed for the motion map based post-processing step to further reduce the false positives that are brought by semantic noise. The experiments on both CDnet2014 and Lasiesta show that our method outperforms many state-of-the-art methods, quantitively and qualitatively.

Foreground Segmentation Using Multiscale Convolutional Neural Network

Adaptive Foreground and Shadow Segmentation Using Hidden Conditional Random Fields

Foreground Gating and Background Refining Network for Surveillance Object Detection

Foreground Detection in Surveillance Video with Fully Convolutional Semantic Network

A Universal Foreground Segmentation Technique using Deep-Neural Network

Foreground Detection Based on Superpixel and Semantic Segmentation

Robust Bilayer Segmentation and Motion/Depth Estimation with a Handheld Camera

Foreground segmentation based on multi-resolution and matting.

Deep Features Homography Transformation Fusion Network-A Universal Foreground Segmentation Algorithm for PTZ Cameras and a Comparative Study

MFCP-Net: Multiscale Feature Correlation Perception Network for Image Segmentation Based on Background Prior

Integrated Multi-Scale Event Verification In An Augmented Foreground Motion Space

Enhanced Multi-Scale Feature Adaptive Fusion Sparse Convolutional Network for Large-Scale Scenes Semantic Segmentation

Multi-Channel Adaptive Mixture Background Model for Real-time Tracking.

Pixel Matching Network for Cross-Domain Few-Shot Segmentation

Motion-Guided Cascaded Refinement Network for Video Object Segmentation

Remote Sensing Image Semantic Segmentation Method Based on a Deep Convolutional Neural Network and Multiscale Feature Fusion

An Effective Method for Foreground Segmentation of Video

Efficient Background Model Based on Multi-Level Feedback for Video Surveillance

Automatic foreground segmentation using light field images

Research on Fine-Grained Image Recognition Based on Foreground Extraction and Convolutional Neural Networks

Background subtraction for video sequence using deep neural network