Abstract:Adverse weather removal tasks like deraining, desnowing, and dehazing are usually treated as separate tasks. However, in practical autonomous driving scenarios, the type, intensity,and mixing degree of weather are unknown, so handling each task separately cannot deal with the complex practical scenarios. In this paper, we study the blind adverse weather removal problem. Mixture-of-Experts (MoE) is a popular model that adopts a learnable gate to route the input to different expert networks. The principle of MoE involves using adaptive networks to process different types of unknown inputs. Therefore, MoE has great potential for blind adverse weather removal. However, the original MoE module is inadequate for coupled multiple weather types and fails to utilize multi-scale features for better performance. To this end, we propose a method called Weather-aware Multi-scale MoE (WM-MoE) based on Transformer for blind weather removal. WM-MoE includes two key designs: WEather-Aware Router (WEAR) and Multi-Scale Experts (MSE). WEAR assigns experts for each image token based on decoupled content and weather features, which enhances the model's capability to process multiple adverse weathers. To obtain discriminative weather features from images, we propose Weather Guidance Fine-grained Contrastive Learning (WGF-CL), which utilizes weather cluster information to guide the assignment of positive and negative samples for each image token. Since processing different weather types requires different receptive fields, MSE leverages multi-scale features to enhance the spatial relationship modeling capability, facilitating the high-quality restoration of diverse weather types and intensities. Our method achieves state-of-the-art performance in blind adverse weather removal on two public datasets and our dataset. We also demonstrate the advantage of our method on downstream segmentation tasks.

Weather Translation Via Weather-Cue Transferring.

Weather GAN: Multi-Domain Weather Translation Using Generative Adversarial Networks

Adverse Weather Image Translation with Asymmetric and Uncertainty-aware GAN

UATST: Towards Unpaired Arbitrary Text-Guided Style Transfer with Cross-Space Modulation

A Multi-Task Framework for Weather Recognition

Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal

Weather-oriented Domain Generalization of Semantic Segmentation for Autonomous Driving.

A CNN-based Multi-Task Framework for Weather Recognition with Multi-Scale Weather Cues

Analogical Image Translation for Fog Generation

Spatio-Temporal Enhanced Contrastive and Contextual Learning for Weather Forecasting

WM-MoE: Weather-aware Multi-scale Mixture-of-Experts for Blind Adverse Weather Removal

WeatherProof: Leveraging Language Guidance for Semantic Segmentation in Adverse Weather

Generation of Multi-Spectral Scene Images under Different Weather Conditions

ForkGAN: Seeing into the Rainy Night

Multi-Weather Degradation-Aware Transformer for Image Restoration

Modeling Weather Uncertainty for Multi-weather Co-Presence Estimation

Mutual-GAN: Towards Unsupervised Cross-Weather Adaptation with Mutual Information Constraint

Two-Class Weather Classification

Advancing Meteorological Forecasting: AI-based Approach to Synoptic Weather Map Analysis

Weatherproofing Retrieval for Localization with Generative AI and Geometric Consistency