Abstract:In this paper, we address the unified image restoration challenge by reframing it as a contrastive learning-based classification problem. Despite the significant strides made by deep learning methods in enhancing image restoration quality, their limited capacity to generalize across diverse degradation types and intensities necessitates the training of separate models for each specific degradation scenario. We proposes an all-encompassing approach that can restore images from various unknown corruption types and levels. We devise a method that learns representations of the latent sharp image’s degradation and accompanying textual features (such as dataset categories and image content descriptions), converting these into prompts which are then embedded within a reconstruction network model to enhance cross-database restoration performance. This culminates in a unified image reconstruction framework. The study involves two stages: In the first stage, we design a MultiContentNet that learns multi-modal features (MMFs) of the latent sharp image. This network encodes the visual degradation expressions and contextual text features into latent variables, thereby exerting a guided classification effect. Specifically, MultiContentNet is trained as an auxiliary controller capable of taking the degraded input image and, through contrastive learning, extracts MMFs of the latent target image. This effectively generates natural classifiers tailored for different degradation types. The second phase integrates the learned MMFs into an image restoration network via cross-attention mechanisms. This guides the restoration model to learn high-fidelity image recovery. Experiments conducted on six blind image restoration tasks demonstrate that the proposed method achieves state-of-the-art performance, highlighting the potential significance of large-scale pretrained vision-language models’ MMFs in advancing high-quality unified image reconstruction.

Degradation-Aware Blind Face Restoration Via High-Quality VQ Codebook.

Extreme-Quality Computational Imaging via Degradation Framework

VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decoder

Blind restoration of compressed degraded image based on block effect suppression

DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration

Degradation learning and Skip-Transformer for blind face restoration

RestoreFormer: High-Quality Blind Face Restoration from Undegraded Key-Value Pairs

Towards Robust Blind Face Restoration with Codebook Lookup Transformer

Learning Degradation-unaware Representation with Prior-based Latent Transformations for Blind Face Restoration

Blind Face Video Restoration with Temporal Consistent Generative Prior and Degradation-Aware Prompt

From Face to Natural Image: Learning Real Degradation for Blind Image Super-Resolution

DifFace: Blind Face Restoration with Diffused Error Contraction

TFRGAN: Leveraging Text Information for Blind Face Restoration with Extreme Degradation.

LD-BFR: Vector-Quantization-Based Face Restoration Model with Latent Diffusion Enhancement

Efficient Video Face Enhancement with Enhanced Spatial-Temporal Consistency

Degradation Oriented and Regularized Network for Blind Depth Super-Resolution

Exploring Correlations in Degraded Spatial Identity Features for Blind Face Restoration

Multi-modal Degradation Feature Learning for Unified Image Restoration Based on Contrastive Learning

Enhanced Blind Face Restoration with Multi-Exemplar Images and Adaptive Spatial Feature Fusion

Semantic-Aware Detail Enhancement for Blind Face Restoration