Abstract:Video rain/snow removal from surveillance videos is an important task in the computer vision community since rain/snow existed in videos can severely degenerate the performance of many surveillance system. Various methods have been investigated extensively, but most only consider consistent rain/snow under stable background scenes. Rain/snow captured from practical surveillance camera, however, is always highly dynamic in time, and those videos also include occasionally transformed background scenes and background motions caused by waving leaves or water surfaces. To this issue, this paper proposes a novel rain/snow removal approach, which fully considers dynamic statistics of both rain/snow and background scenes taken from a video sequence. Specifically, the rain/snow is encoded as an online multi-scale convolutional sparse coding (OMS-CSC) model, which not only finely delivers the sparse scattering and multi-scale shapes of real rain/snow, but also well distinguish the components of background motion from rain/snow layer. The real-time ameliorated parameters in the model well encodes their temporally dynamic configurations. Furthermore, a transformation operator imposed on the background scenes is further embedded into the proposed model, which finely conveys the background transformations, such as rotations, scalings and distortions, inevitably existed in a real video sequence. The approach so constructed can naturally better adapt to the dynamic rain/snow as well as background changes, and also suitable to deal with the streaming video attributed its online learning mode. The proposed model is formulated in a concise maximum a posterior (MAP) framework and is readily solved by the alternating direction method of multipliers (ADMM). Compared with the state-of-the-art online and offline video rain/snow removal methods, the proposed method achieves best performance on synthetic and real videos datasets both visually and quantitatively. Specifically, our method can-be implemented in relatively high efficiency, showing its potential to real-time video rain/snow removal. The code page is at: https://github.com/MinghanLi/OTMSCSC_matlab_2020.

Depth prior-based stable tensor decomposition for video snow removal

Video Desnowing and Deraining via Saliency and Dual Adaptive Spatiotemporal Filtering

Video Snow Removal Based on Self-Adaptation Snow Detection and Patch-Based Gaussian Mixture Model

Video Rain/Snow Removal by Transformed Online Multiscale Convolutional Sparse Coding

Online Rain/Snow Removal From Surveillance Videos

Deep Dense Multi-Scale Network for Snow Removal Using Semantic and Depth Priors

ALL Snow Removed - Single Image Desnowing Algorithm Using Hierarchical Dual-tree Complex Wavelet Representation and Contradict Channel Loss.

JSTASR: Joint Size and Transparency-Aware Snow Removal Algorithm Based on Modified Partial Convolution and Veiling Effect Removal

MSP-Former: Multi-Scale Projection Transformer for Single Image Desnowing

Towards Real-time High-Definition Image Snow Removal: Efficient Pyramid Network with Asymmetrical Encoder-decoder Architecture

Snow Fluff Detection and Removal from Video Images

Deep Dense Multi-scale Network for Snow Removal Using Semantic and Geometric Priors

Detecting and Extracting Natural Snow from Videos

Real-Time Rendering of Large-Scale Snow Scene

Semi-Supervised Video Desnowing Network via Temporal Decoupling Experts and Distribution-Driven Contrastive Regularization

A Closed-Form Solution to Video Matting of Natural Snow

Spatio-Temporal Depth Recovery of Dynamic Scenes with Multiple Handheld Cameras

DesnowNet: Context-Aware Deep Network for Snow Removal

Image Desnowing via Deep Invertible Separation

Depth Propagation with Tensor Voting for 2D-to-3d Video Conversion

Falling Snow Motion Estimation Based on a Semi-Transparent and Particle Trajectory Model