Abstract:Adverse weather removal tasks like deraining, desnowing, and dehazing are usually treated as separate tasks. However, in practical autonomous driving scenarios, the type, intensity,and mixing degree of weather are unknown, so handling each task separately cannot deal with the complex practical scenarios. In this paper, we study the blind adverse weather removal problem. Mixture-of-Experts (MoE) is a popular model that adopts a learnable gate to route the input to different expert networks. The principle of MoE involves using adaptive networks to process different types of unknown inputs. Therefore, MoE has great potential for blind adverse weather removal. However, the original MoE module is inadequate for coupled multiple weather types and fails to utilize multi-scale features for better performance. To this end, we propose a method called Weather-aware Multi-scale MoE (WM-MoE) based on Transformer for blind weather removal. WM-MoE includes two key designs: WEather-Aware Router (WEAR) and Multi-Scale Experts (MSE). WEAR assigns experts for each image token based on decoupled content and weather features, which enhances the model's capability to process multiple adverse weathers. To obtain discriminative weather features from images, we propose Weather Guidance Fine-grained Contrastive Learning (WGF-CL), which utilizes weather cluster information to guide the assignment of positive and negative samples for each image token. Since processing different weather types requires different receptive fields, MSE leverages multi-scale features to enhance the spatial relationship modeling capability, facilitating the high-quality restoration of diverse weather types and intensities. Our method achieves state-of-the-art performance in blind adverse weather removal on two public datasets and our dataset. We also demonstrate the advantage of our method on downstream segmentation tasks.

A Multi-Task Framework for Weather Recognition

A CNN-based Multi-Task Framework for Weather Recognition with Multi-Scale Weather Cues

A CNN-RNN Architecture for Multi-Label Weather Recognition

Multi-Task Collaboration Deep Learning Framework for Infrared Precipitation Estimation.

Multi-class Weather Classification Fusing Weather Dataset and Image Features

All-weather Multi-Modality Image Fusion: Unified Framework and 100k Benchmark

Modeling Weather Uncertainty for Multi-weather Co-Presence Estimation

A Study of Weather-Image Classification Combining VIT and a Dual Enhanced-Attention Module

When Self-Supervised Learning Meets Scene Classification: Remote Sensing Scene Classification Based on a Multitask Learning Framework

Multiple weather images restoration using the task transformer and adaptive mixup strategy

Multi‐task Framework of Precipitation Nowcasting

MASK-CNN-Transformer for real-time multi-label weather recognition

An Adaptive Multimodal Fusion 3D Object Detection Algorithm for Unmanned Systems in Adverse Weather

Weather-aware autopilot: Domain generalization for point cloud semantic segmentation in diverse weather scenarios

Scene recognition under special traffic conditions based on deep multi‐task learning

WM-MoE: Weather-aware Multi-scale Mixture-of-Experts for Blind Adverse Weather Removal

MMSTP: Multi-modal Spatiotemporal Feature Fusion Network for Precipitation Prediction

Confidence-Enhanced Mutual Knowledge for Uncertain Segmentation

RSCM: Region Selection and Concurrency Model for Multi-Class Weather Recognition.

Weather Translation Via Weather-Cue Transferring.

Spatio-Temporal Enhanced Contrastive and Contextual Learning for Weather Forecasting