Abstract: In this paper, we address a rain removal problem from a single image, even in the presence of heavy rain and rain streak accumulation. Our core ideas lie in the new rain image models and a novel deep learning architecture. We first modify an existing model comprising a rain streak layer and a background layer, by adding a binary map that locates rain streak regions. Second, we create a new model consisting of a component representing rain streak accumulation (where individual streaks cannot be seen, and thus visually similar to mist or fog), and another component representing various shapes and directions of overlapping rain streaks, which usually happen in heavy rain. Based on the first model, we develop a multi-task deep learning architecture that learns the binary rain streak map, the appearance of rain streaks, and the clean background, which is our ultimate output. The additional binary map is critically beneficial, since its loss function can provide additional strong information to the network. To handle rain streak accumulation (again, a phenomenon visually similar to mist or fog) and various shapes and directions of overlapping rain streaks, we propose a recurrent rain detection and removal network that removes rain streaks and clears up the rain accumulation iteratively and progressively. In each recurrence of our method, a new contextualized dilated network is developed to exploit regional contextual information and outputs better representation for rain detection. The evaluation on real images, particularly on heavy rain, shows the effectiveness of our novel models and architecture, outperforming the state-of-the-art methods significantly. Our codes and data sets will be publicly available.

A Summary of Research Progress of Single Image to Remove Rain and Fog Based on Deep Learning

A Survey of Single Image Rain Removal Based on Deep Learning

A Comprehensive Review of Traditional and Deep-Learning-Based Defogging Algorithms

Deep Single Image Deraining Via Estimating Transmission and Atmospheric Light in rainy Scenes

Deep Single Image Deraining Via Modeling Haze-Like Effect.

An investigation of rain streak removal models based on expert experience and deep learning

Vehicle-Related Scene Understanding Using Deep Learning

Deep Vision in Analysis and Recognition of Radar Data: Achievements, Advancements and Challenges

A Comprehensive Survey on Image Dehazing Based on Deep Learning.

Deep Learning-Based Atmospheric Visibility Detection

Optimization and Application of Image Defogging Algorithm Based on Deep Learning Network

Deep Joint Rain Detection and Removal from a Single Image

A Comprehensive Survey and Taxonomy on Single Image Dehazing Based on Deep Learning

Based on single image defogging algorithm

Single image rain streaks removal: a review and an exploration

A Haze Removal Method Based on Depth Information Fusion

Deep joint rain and haze removal from single images

Decomposition Makes Better Rain Removal: An Improved Attention-Guided Deraining Network

Research and Implementation of Image Rain Removal Based on Deep Learning

Joint rain and atmospheric veil removal from single image