Abstract:In this paper, we address the unified image restoration challenge by reframing it as a contrastive learning-based classification problem. Despite the significant strides made by deep learning methods in enhancing image restoration quality, their limited capacity to generalize across diverse degradation types and intensities necessitates the training of separate models for each specific degradation scenario. We proposes an all-encompassing approach that can restore images from various unknown corruption types and levels. We devise a method that learns representations of the latent sharp image’s degradation and accompanying textual features (such as dataset categories and image content descriptions), converting these into prompts which are then embedded within a reconstruction network model to enhance cross-database restoration performance. This culminates in a unified image reconstruction framework. The study involves two stages: In the first stage, we design a MultiContentNet that learns multi-modal features (MMFs) of the latent sharp image. This network encodes the visual degradation expressions and contextual text features into latent variables, thereby exerting a guided classification effect. Specifically, MultiContentNet is trained as an auxiliary controller capable of taking the degraded input image and, through contrastive learning, extracts MMFs of the latent target image. This effectively generates natural classifiers tailored for different degradation types. The second phase integrates the learned MMFs into an image restoration network via cross-attention mechanisms. This guides the restoration model to learn high-fidelity image recovery. Experiments conducted on six blind image restoration tasks demonstrate that the proposed method achieves state-of-the-art performance, highlighting the potential significance of large-scale pretrained vision-language models’ MMFs in advancing high-quality unified image reconstruction.

Prior embedding multi-degradations super resolution network

Degradation-Guided Multi-Modal Fusion Network for Depth Map Super-Resolution

Hyperspectral Image Super-resolution with Deep Priors and Degradation Model Inversion

Multi-modal Degradation Feature Learning for Unified Image Restoration Based on Contrastive Learning

A method of degradation mechanism-based unsupervised remote sensing image super-resolution

Multi-Source Deep Residual Fusion Network for Depth Image Super-resolution

Conditional Hyper-Network for Blind Super-Resolution with Multiple Degradations

Meta-Learning based Degradation Representation for Blind Super-Resolution

Degradation-Guided Meta-Restoration Network for Blind Super-Resolution

DeeDSR: Towards Real-World Image Super-Resolution via Degradation-Aware Stable Diffusion

Double Prior Network for Multidegradation Remote Sensing Image Super-Resolution

Meta-learning based blind image super-resolution approach to different degradations

Learning a Single Convolutional Super-Resolution Network for Multiple Degradations

Adaptive Blind Super-Resolution Network for Spatial-Specific and Spatial-Agnostic Degradations

Degradation Oriented and Regularized Network for Blind Depth Super-Resolution

Multi-Stream Fusion Network for Multi-Distortion Image Super-Resolution

A practical super-resolution method for multi-degradation remote sensing images with deep convolutional neural networks

Hyperspectral Reconstruction From RGB Images via Physically Guided Graph Deep Prior Learning

MAIN: Multibranch Attention Integration Network for Degraded Remote-Sensing Image Super-Resolution

Difficulty-Guided Variant Degradation Learning for Blind Image Super-Resolution

Unsupervised image blind super resolution via real degradation feature learning