Abstract:In this paper, we address a rain removal problem from a single image, even in the presence of heavy rain and rain accumulation. Our core ideas lie in our new rain image models and a novel deep learning architecture. We first modify the commonly used model, which is a linear combination of a rain streak layer and a background layer, by adding a binary map that locates rain streak regions. Second, we create a model consisting of a component representing rain accumulation (where individual streaks cannot be seen, and thus visually similar to mist or fog), and another component representing various shapes and directions of overlapping rain streaks, which normally happen in heavy rain. Based on the first model, we develop a multi-task deep learning architecture that learns the binary rain streak map, the appearance of rain streaks, and the clean background, which is our ultimate output. The additional binary map is critically beneficial, since its loss function can provide additional strong information to the network. In many cases though, rain streaks can be dense and large in their size, thus to obtain the clean background, we need spatial contextual information. For this, we utilize the dilated convolution. To handle rain accumulation (again, a phenomenon visually similar to mist or fog) and various shapes and directions of overlapping rain streaks, we propose an iterative information feedback (IIF) network that removes rain streaks and clears up the rain accumulation iteratively and progressively. Overall, this multi-task learning and iterative information feedback benefits each other and constitutes a network that is end-to-end trainable. Our extensive evaluation on real images, particularly on heavy rain, shows the effectiveness of our novel models and architecture, outperforming the state-of-the-art methods significantly.

Incremental Image De-raining via Associative Memory

Image De-raining via Continual Learning

Improving De-raining Generalization Via Neural Reorganization

Joint Rain Detection and Removal via Iterative Region Dependent Multi-Task Learning.

On the Exploration of Incremental Learning for Fine-grained Image Retrieval

Continual Image Deraining with Hypergraph Convolutional Networks.

An Efficient Strategy for Catastrophic Forgetting Reduction in Incremental Learning

Single Image Deraining Using a Recurrent Multi-scale Aggregation and Enhancement Network

Continual Recognition with Adaptive Memory Update.

Multi-Level Memory Compensation Network for Rain Removal via Divide-and-Conquer Strategy

Recurrent Multi-Frame Deraining: Combining Physics Guidance and Adversarial Learning

Incremental Concept Learning via Online Generative Memory Recall

Incremental Scene Classification Using Dual Knowledge Distillation and Classifier Discrepancy on Natural and Remote Sensing Images

Scale-Free Single Image Deraining Via Visibility-Enhanced Recurrent Wavelet Learning

Single image de-raining via clique recursive feedback mechanism

DILRS: Domain-Incremental Learning for Semantic Segmentation in Multi-Source Remote Sensing Data

Continual All-in-One Adverse Weather Removal With Knowledge Replay on a Unified Network Structure

Single-Image Deraining via Recurrent Residual Multiscale Networks

Incremental Open-set Domain Adaptation

Incremental Learning Via Rate Reduction

An Efficient Single Image De-Raining Model With Decoupled Deep Networks