Abstract:In this paper, we address the unified image restoration challenge by reframing it as a contrastive learning-based classification problem. Despite the significant strides made by deep learning methods in enhancing image restoration quality, their limited capacity to generalize across diverse degradation types and intensities necessitates the training of separate models for each specific degradation scenario. We proposes an all-encompassing approach that can restore images from various unknown corruption types and levels. We devise a method that learns representations of the latent sharp image’s degradation and accompanying textual features (such as dataset categories and image content descriptions), converting these into prompts which are then embedded within a reconstruction network model to enhance cross-database restoration performance. This culminates in a unified image reconstruction framework. The study involves two stages: In the first stage, we design a MultiContentNet that learns multi-modal features (MMFs) of the latent sharp image. This network encodes the visual degradation expressions and contextual text features into latent variables, thereby exerting a guided classification effect. Specifically, MultiContentNet is trained as an auxiliary controller capable of taking the degraded input image and, through contrastive learning, extracts MMFs of the latent target image. This effectively generates natural classifiers tailored for different degradation types. The second phase integrates the learned MMFs into an image restoration network via cross-attention mechanisms. This guides the restoration model to learn high-fidelity image recovery. Experiments conducted on six blind image restoration tasks demonstrate that the proposed method achieves state-of-the-art performance, highlighting the potential significance of large-scale pretrained vision-language models’ MMFs in advancing high-quality unified image reconstruction.

Cycle contrastive adversarial learning with structural consistency for unsupervised high-quality image deraining transformer

Cycle Contrastive Adversarial Learning for Unsupervised image Deraining

Unpaired Deep Image Deraining Using Dual Contrastive Learning

Unsupervised Deraining: Where Contrastive Learning Meets Self-similarity

Contrastive Unfolding Deraining Network

Unsupervised Deraining: Where Asymmetric Contrastive Learning Meets Self-similarity

Combining multiscale learning and attention mechanism densely connected network for single image deraining

DerainCycleGAN: Rain Attentive CycleGAN for Single Image Deraining and Rainmaking

Deep Single Image Deraining using An Asymetric Cycle Generative and Adversarial Framework

CTFCD: Channel Transformer Based on Full Convolutional Decoder for Single Image Deraining

Deep single image deraining using an asymmetric cyclic generative and adversarial framework

Hybrid CNN-Transformer Feature Fusion for Single Image Deraining

Contrastive Learning Based Recursive Dynamic Multi-Scale Network for Image Deraining

A Hybrid CNN-Transformer Architecture with Frequency Domain Contrastive Learning for Image Deraining

Local and global knowledge distillation with direction-enhanced contrastive learning for single-image deraining

Cycle-attention-derain: unsupervised rain removal with CycleGAN

Enhanced Attentive Generative Adversarial Network for Single-Image Deraining.

Multi-modal Degradation Feature Learning for Unified Image Restoration Based on Contrastive Learning

Multi-Scale Dilated Convolution Transformer for Single Image Deraining

Harnessing Joint Rain-/Detail-aware Representations to Eliminate Intricate Rains

Online-updated High-order Collaborative Networks for Single Image Deraining