Abstract:Removing adverse weather conditions such as rain, raindrop, and snow from images is critical for various real-world applications, including autonomous driving, surveillance, and remote sensing. However, existing multi-task approaches typically rely on augmenting the model with additional parameters to handle multiple scenarios. While this enables the model to address diverse tasks, the introduction of extra parameters significantly complicates its practical deployment. In this paper, we propose a novel Gradient-Guided Parameter Mask for Multi-Scenario Image Restoration under adverse weather, designed to effectively handle image degradation under diverse weather conditions without additional parameters. Our method segments model parameters into common and specific components by evaluating the gradient variation intensity during training for each specific weather condition. This enables the model to precisely and adaptively learn relevant features for each weather scenario, improving both efficiency and effectiveness without compromising on performance. This method constructs specific masks based on gradient fluctuations to isolate parameters influenced by other tasks, ensuring that the model achieves strong performance across all scenarios without adding extra parameters. We demonstrate the state-of-the-art performance of our framework through extensive experiments on multiple benchmark datasets. Specifically, our method achieves PSNR scores of 29.22 on the Raindrop dataset, 30.76 on the Rain dataset, and 29.56 on the Snow100K dataset. Code is available at: \href{<a class="link-external link-https" href="https://github.com/AierLab/MultiTask" rel="external noopener nofollow">this https URL</a>}{<a class="link-external link-https" href="https://github.com/AierLab/MultiTask" rel="external noopener nofollow">this https URL</a>}.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively handle image degradation without adding extra parameters when performing multi - scene image inpainting under various adverse weather conditions (such as rain, raindrops and snow). Specifically: 1. **Limitations of existing methods**: - Existing multi - task methods usually handle multiple scenes by adding extra parameters to the model. Although this enables the model to cope with diverse tasks, it significantly increases the complexity of actual deployment. - Other methods often need to optimize the model separately for each weather condition when dealing with image degradation under different weather conditions, resulting in low efficiency. 2. **Proposed new method**: - This paper proposes a method based on Gradient - Guided Parameter Mask, aiming to effectively handle image degradation problems under various adverse weather conditions without adding extra parameters. - This method divides the model parameters into general components and specific components by analyzing the intensity of gradient changes under each specific weather condition during the training process, so that the model can accurately and adaptively learn the relevant features of each weather scene, improving efficiency and effectiveness. 3. **Specific implementation**: - **Parameter segmentation**: According to the intensity of gradient changes, the model parameters are divided into general parameters and task - specific parameters. General parameters capture the consistent features under all weather conditions, while task - specific parameters adapt to the unique characteristics of each specific weather scene. - **Gradient - guided mask determination**: Calculate the gradient of each parameter with respect to the task - specific loss through back - propagation, and select the top 10% of parameters with the highest absolute value of the gradient as task - specific parameters. - **Selective parameter training**: Only train the task - specific parameters identified by the mask while keeping the general parameters frozen, thus ensuring that the model maintains computational efficiency while adapting to specific weather conditions. 4. **Experimental results**: - Through extensive experiments on multiple benchmark datasets, it is proved that this method has achieved state - of - the - art performance in image inpainting tasks under various adverse weather conditions such as rain, raindrops and snow. For example, the PSNR score is 29.22 on the Raindrop dataset, 30.76 on the Rain dataset, and 29.56 on the Snow100K dataset. In conclusion, the main goal of this paper is to solve the efficiency and performance problems in multi - scene image inpainting under various adverse weather conditions by introducing the Gradient - Guided Parameter Mask method, while avoiding the complexity brought by adding extra parameters.

Gradient-Guided Parameter Mask for Multi-Scenario Image Restoration Under Adverse Weather

Learning Weather-General and Weather-Specific Features for Image Restoration under Multiple Adverse Weather Conditions

Always Clear Days: Degradation Type and Severity Aware All-In-One Adverse Weather Removal

Multiple weather images restoration using the task transformer and adaptive mixup strategy

WM-MoE: Weather-aware Multi-scale Mixture-of-Experts for Blind Adverse Weather Removal

A New Interactive Enhanced Method for Image with Poor Visibility

Restoring Images Captured in Arbitrary Hybrid Adverse Weather Conditions in One Go

Multi-Weather Degradation-Aware Transformer for Image Restoration

Multi-Weather Image Restoration via Histogram-Based Transformer Feature Enhancement

Exploring the Application of Large-Scale Pre-Trained Models on Adverse Weather Removal

Snow Mask Guided Adaptive Residual Network for image snow removal

TANet: Triplet Attention Network for All-In-One Adverse Weather Image Restoration

ACMamba: A State Space Model-Based Approach for Multi-Weather Degraded Image Restoration

MWFormer: Multi-Weather Image Restoration Using Degradation-Aware Transformers

Generation of Multi-Spectral Scene Images under Different Weather Conditions

All-weather Multi-Modality Image Fusion: Unified Framework and 100k Benchmark

Rainmer: Learning Multi-view Representations for Comprehensive Image Deraining and Beyond

Image De-raining via Continual Learning

Parameter Efficient Adaptation for Image Restoration with Heterogeneous Mixture-of-Experts

MetaWeather: Few-Shot Weather-Degraded Image Restoration