Abstract:In this paper, we address the unified image restoration challenge by reframing it as a contrastive learning-based classification problem. Despite the significant strides made by deep learning methods in enhancing image restoration quality, their limited capacity to generalize across diverse degradation types and intensities necessitates the training of separate models for each specific degradation scenario. We proposes an all-encompassing approach that can restore images from various unknown corruption types and levels. We devise a method that learns representations of the latent sharp image’s degradation and accompanying textual features (such as dataset categories and image content descriptions), converting these into prompts which are then embedded within a reconstruction network model to enhance cross-database restoration performance. This culminates in a unified image reconstruction framework. The study involves two stages: In the first stage, we design a MultiContentNet that learns multi-modal features (MMFs) of the latent sharp image. This network encodes the visual degradation expressions and contextual text features into latent variables, thereby exerting a guided classification effect. Specifically, MultiContentNet is trained as an auxiliary controller capable of taking the degraded input image and, through contrastive learning, extracts MMFs of the latent target image. This effectively generates natural classifiers tailored for different degradation types. The second phase integrates the learned MMFs into an image restoration network via cross-attention mechanisms. This guides the restoration model to learn high-fidelity image recovery. Experiments conducted on six blind image restoration tasks demonstrate that the proposed method achieves state-of-the-art performance, highlighting the potential significance of large-scale pretrained vision-language models’ MMFs in advancing high-quality unified image reconstruction.

Multi-Weather Degradation-Aware Transformer for Image Restoration

MWFormer: Multi-Weather Image Restoration Using Degradation-Aware Transformers

Multi-Weather Image Restoration via Histogram-Based Transformer Feature Enhancement

Learning Weather-General and Weather-Specific Features for Image Restoration under Multiple Adverse Weather Conditions

UTDM: a universal transformer-based diffusion model for multi-weather-degraded images restoration

Always Clear Days: Degradation Type and Severity Aware All-In-One Adverse Weather Removal

Multiple weather images restoration using the task transformer and adaptive mixup strategy

Towards an Effective and Efficient Transformer for Rain-by-snow Weather Removal

ACMamba: A State Space Model-Based Approach for Multi-Weather Degraded Image Restoration

Decoupling Degradation and Content Processing for Adverse Weather Image Restoration

GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions

Degradation Type-Aware Image Restoration for Effective Object Detection in Adverse Weather

Restoring Images in Adverse Weather Conditions via Histogram Transformer

Multi-modal Degradation Feature Learning for Unified Image Restoration Based on Contrastive Learning

An Efficient Dehazing Algorithm Based on the Fusion of Transformer and Convolutional Neural Network.

MetaWeather: Few-Shot Weather-Degraded Image Restoration

Deep Fuzzy Clustering Transformer: Learning the General Property of Corruptions for Degradation-Agnostic Multi-Task Image Restoration

Joint multi-dimensional dynamic attention and transformer for general image restoration

AllRestorer: All-in-One Transformer for Image Restoration under Composite Degradations

Image De-Raining Transformer

OneRestore: A Universal Restoration Framework for Composite Degradation