Abstract:In this paper, we address a rain removal problem from a single image, even in the presence of heavy rain and rain accumulation. Our core ideas lie in our new rain image models and a novel deep learning architecture. We first modify the commonly used model, which is a linear combination of a rain streak layer and a background layer, by adding a binary map that locates rain streak regions. Second, we create a model consisting of a component representing rain accumulation (where individual streaks cannot be seen, and thus visually similar to mist or fog), and another component representing various shapes and directions of overlapping rain streaks, which normally happen in heavy rain. Based on the first model, we develop a multi-task deep learning architecture that learns the binary rain streak map, the appearance of rain streaks, and the clean background, which is our ultimate output. The additional binary map is critically beneficial, since its loss function can provide additional strong information to the network. In many cases though, rain streaks can be dense and large in their size, thus to obtain the clean background, we need spatial contextual information. For this, we utilize the dilated convolution. To handle rain accumulation (again, a phenomenon visually similar to mist or fog) and various shapes and directions of overlapping rain streaks, we propose an iterative information feedback (IIF) network that removes rain streaks and clears up the rain accumulation iteratively and progressively. Overall, this multi-task learning and iterative information feedback benefits each other and constitutes a network that is end-to-end trainable. Our extensive evaluation on real images, particularly on heavy rain, shows the effectiveness of our novel models and architecture, outperforming the state-of-the-art methods significantly.

Multi-Scale Features Joint Rain Removal for Single Image.

From Coarse to Fine: A Two-Stage Network for Dense Haze removal

Multi-scale Fusion Residual Network For Single Image Rain Removal

DCSFN: Deep Cross-scale Fusion Network for Single Image Rain Removal

See clearly on rainy days: Hybrid multiscale loss guided multi-feature fusion network for single image rain removal

Multi-scale Single Image Rain Removal Using a Squeeze-and-excitation Residual Network.

Multi-Scale Progressive Fusion Network for Single Image Deraining

Joint Rain Detection and Removal from a Single Image with Contextualized Deep Networks.

Single image deraining using multi‐stage and multi‐scale joint channel coordinate attention fusion network

Multi-scale Features Fusion Network for Single Image Deraining

Single-image de-raining using joint filter and multi-scale deep alternate-connection dense network

Multi-Scale Hourglass Hierarchical Fusion Network for Single Image Deraining

Combining multiscale learning and attention mechanism densely connected network for single image deraining

Multi-Scale Hybrid Fusion Network for Single Image Deraining

Deep joint rain and haze removal from single images

Deep Joint Rain Detection and Removal from a Single Image

Joint Rain Detection and Removal via Iterative Region Dependent Multi-Task Learning.

A Joint De-Rain and De-Mist Network Based on the Atmospheric Scattering Model

Research on Rain Removal Method for Single Image Based on Multi-channel and Multi-scale CNN

Joint rain and atmospheric veil removal from single image