Abstract:Since rainy weather always degrades image quality and poses significant challenges to most computer vision-based intelligent systems, image de-raining has been a hot research topic. Fortunately, in a rainy light field (LF) image, background obscured by rain streaks in one sub-view may be visible in the other sub-views, and implicit depth information and recorded 4D structural information may benefit rain streak detection and removal. However, existing LF image rain removal methods either do not fully exploit the global correlations of 4D LF data or only utilize partial sub-views, resulting in sub-optimal rain removal performance and no-equally good quality for all de-rained sub-views. In this paper, we propose an efficient network, called MDeRainNet, for rain streak removal from LF images. The proposed network adopts a multi-scale encoder-decoder architecture, which directly works on Macro-pixel images (MPIs) to improve the rain removal performance. To fully model the global correlation between the spatial and the angular information, we propose an Extended Spatial-Angular Interaction (ESAI) module to merge them, in which a simple and effective Transformer-based Spatial-Angular Interaction Attention (SAIA) block is also proposed for modeling long-range geometric correlations and making full use of the angular information. Furthermore, to improve the generalization performance of our network on real-world rainy scenes, we propose a novel semi-supervised learning framework for our MDeRainNet, which utilizes multi-level KL loss to bridge the domain gap between features of synthetic and real-world rain streaks and introduces colored-residue image guided contrastive regularization to reconstruct rain-free images. Extensive experiments conducted on synthetic and real-world LFIs demonstrate that our method outperforms the state-of-the-art methods both quantitatively and qualitatively.

Multi-patch de-raindrop Transformer for UAV images

Multi-Scale Hybrid Fusion Network for Single Image Deraining

Image De-Raining Transformer

Multi-Scale Progressive Fusion Network for Single Image Deraining

Not All Areas Are Equal: A Novel Separation‐Restoration‐Fusion Network for Image Raindrop Removal

Multi-Scale Fusion and Decomposition Network for Single Image Deraining

Frequency-oriented hierarchical fusion network for single image raindrop removal

Hybrid CNN-Transformer Feature Fusion for Single Image Deraining

DR-DiT: Image Deraining Using Diffusion Model with Transformer

An Efficient Dehazing Algorithm Based on the Fusion of Transformer and Convolutional Neural Network.

Sparse Sampling Transformer with Uncertainty-Driven Ranking for Unified Removal of Raindrops and Rain Streaks

Multi-scale Fusion Residual Network For Single Image Rain Removal

A Prior Guided Wavelet-Spatial Dual Attention Transformer Framework for Heavy Rain Image Restoration

Unformer: A Transformer-Based Approach for Adaptive Multi-Scale Feature Aggregation in Underwater Image Enhancement

MDeRainNet: An Efficient Neural Network for Rain Streak Removal from Macro-pixel Images

Single image deraining using multi‐stage and multi‐scale joint channel coordinate attention fusion network

Feature decoupling and reorganization network for single image deraining

A Hybrid Transformer-Mamba Network for Single Image Deraining

Multi-Scale Hourglass Hierarchical Fusion Network for Single Image Deraining

Dual-Path Multi-Scale Transformer for High-Quality Image Deraining

Multi-Patch Hierarchical Transmission Channel Image Dehazing Network Based on Dual Attention Level Feature Fusion