Abstract: In this paper, we address a rain removal problem from a single image, even in the presence of heavy rain and rain streak accumulation. Our core ideas lie in the new rain image models and a novel deep learning architecture. We first modify an existing model comprising a rain streak layer and a background layer, by adding a binary map that locates rain streak regions. Second, we create a new model consisting of a component representing rain streak accumulation (where individual streaks cannot be seen, and thus visually similar to mist or fog), and another component representing various shapes and directions of overlapping rain streaks, which usually happen in heavy rain. Based on the first model, we develop a multi-task deep learning architecture that learns the binary rain streak map, the appearance of rain streaks, and the clean background, which is our ultimate output. The additional binary map is critically beneficial, since its loss function can provide additional strong information to the network. To handle rain streak accumulation (again, a phenomenon visually similar to mist or fog) and various shapes and directions of overlapping rain streaks, we propose a recurrent rain detection and removal network that removes rain streaks and clears up the rain accumulation iteratively and progressively. In each recurrence of our method, a new contextualized dilated network is developed to exploit regional contextual information and outputs better representation for rain detection. The evaluation on real images, particularly on heavy rain, shows the effectiveness of our novel models and architecture, outperforming the state-of-the-art methods significantly. Our codes and data sets will be publicly available.

Rain Streaks Removal For Single Image Via Directional Total Variation Regularization

Multi-directional rain streak removal based on infimal convolution of oscillation TGV

Directional Regularized Tensor Modeling for Video Rain Streaks Removal

Single Image Rain Removal Boosting Via Directional Gradient

FastDeRain: A Novel Video Rain Streak Removal Method Using Directional Gradient Priors

Removing Rain Streaks by a Linear Model

Direction and Residual Awareness Curriculum Learning Network for Rain Streaks Removal

Rain streaks removal from single image based on texture constraint of background scene

Rain Removal By Image Quasi-Sparsity Priors

End-to-end Rain Streak Removal with RAW Images

Enhanced Attentive Generative Adversarial Network for Single-Image Deraining.

Rain Streak Removal for Single Image via Kernel Guided CNN

Deep Joint Rain Detection and Removal from a Single Image

Joint rain and atmospheric veil removal from single image

Scale-Free Single Image Deraining Via Visibility-Enhanced Recurrent Wavelet Learning

Joint Rain Detection and Removal from a Single Image with Contextualized Deep Networks.

Rain Streaks Elimination Using Image Processing Algorithms

Single Image De-Raining with High-Low Frequency Guidance

See clearly on rainy days: Hybrid multiscale loss guided multi-feature fusion network for single image rain removal