Abstract:Restoring images captured under adverse weather conditions is a fundamental task for many computer vision applications. However, most existing weather restoration approaches are only capable of handling a specific type of degradation, which is often insufficient in real-world scenarios, such as rainy-snowy or rainy-hazy weather. Towards being able to address these situations, we propose a multi-weather Transformer, or MWFormer for short, which is a holistic vision Transformer that aims to solve multiple weather-induced degradations using a single, unified architecture. MWFormer uses hyper-networks and feature-wise linear modulation blocks to restore images degraded by various weather types using the same set of learned parameters. We first employ contrastive learning to train an auxiliary network that extracts content-independent, distortion-aware feature embeddings that efficiently represent predicted weather types, of which more than one may occur. Guided by these weather-informed predictions, the image restoration Transformer adaptively modulates its parameters to conduct both local and global feature processing, in response to multiple possible weather. Moreover, MWFormer allows for a novel way of tuning, during application, to either a single type of weather restoration or to hybrid weather restoration without any retraining, offering greater controllability than existing methods. Our experimental results on multi-weather restoration benchmarks show that MWFormer achieves significant performance improvements compared to existing state-of-the-art methods, without requiring much computational cost. Moreover, we demonstrate that our methodology of using hyper-networks can be integrated into various network architectures to further boost their performance. The code is available at: <a class="link-external link-https" href="https://github.com/taco-group/MWFormer" rel="external noopener nofollow">this https URL</a>

Learning Weather-General and Weather-Specific Features for Image Restoration under Multiple Adverse Weather Conditions

Always Clear Days: Degradation Type and Severity Aware All-In-One Adverse Weather Removal

Multi-Weather Degradation-Aware Transformer for Image Restoration

Restoring Images Captured in Arbitrary Hybrid Adverse Weather Conditions in One Go

Multiple weather images restoration using the task transformer and adaptive mixup strategy

Multi-Weather Image Restoration via Histogram-Based Transformer Feature Enhancement

Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal

Gradient-Guided Parameter Mask for Multi-Scenario Image Restoration Under Adverse Weather

Continual All-in-One Adverse Weather Removal With Knowledge Replay on a Unified Network Structure

Towards Real-World Adverse Weather Image Restoration: Enhancing Clearness and Semantics with Vision-Language Models

Image degradation and recovery based on multiple scattering in remote sensing and bad weather condition

Decoupling Degradation and Content Processing for Adverse Weather Image Restoration

MWFormer: Multi-Weather Image Restoration Using Degradation-Aware Transformers

MetaWeather: Few-Shot Weather-Degraded Image Restoration

Language-driven All-in-one Adverse Weather Removal

WM-MoE: Weather-aware Multi-scale Mixture-of-Experts for Blind Adverse Weather Removal

TANet: Triplet Attention Network for All-In-One Adverse Weather Image Restoration

A New Interactive Enhanced Method for Image with Poor Visibility

LEAN: Local-Enhanced Attention Network for Bad Weather Image Restoration