Abstract:In this paper, we address the unified image restoration challenge by reframing it as a contrastive learning-based classification problem. Despite the significant strides made by deep learning methods in enhancing image restoration quality, their limited capacity to generalize across diverse degradation types and intensities necessitates the training of separate models for each specific degradation scenario. We proposes an all-encompassing approach that can restore images from various unknown corruption types and levels. We devise a method that learns representations of the latent sharp image’s degradation and accompanying textual features (such as dataset categories and image content descriptions), converting these into prompts which are then embedded within a reconstruction network model to enhance cross-database restoration performance. This culminates in a unified image reconstruction framework. The study involves two stages: In the first stage, we design a MultiContentNet that learns multi-modal features (MMFs) of the latent sharp image. This network encodes the visual degradation expressions and contextual text features into latent variables, thereby exerting a guided classification effect. Specifically, MultiContentNet is trained as an auxiliary controller capable of taking the degraded input image and, through contrastive learning, extracts MMFs of the latent target image. This effectively generates natural classifiers tailored for different degradation types. The second phase integrates the learned MMFs into an image restoration network via cross-attention mechanisms. This guides the restoration model to learn high-fidelity image recovery. Experiments conducted on six blind image restoration tasks demonstrate that the proposed method achieves state-of-the-art performance, highlighting the potential significance of large-scale pretrained vision-language models’ MMFs in advancing high-quality unified image reconstruction.

Dual Prior Learning for Blind and Blended Image Restoration

Multi-frame Blind Deconvolution Using Sparse Priors

Implicit Subspace Prior Learning for Dual-Blind Face Restoration

Non-blind image deconvolution using natural image gradient prior

A Double Feature Fusion Network with Progressive Learning for Sharper Inpainting

Towards Unsupervised Blind Face Restoration using Diffusion Prior

BP-DIP: A Backprojection based Deep Image Prior

Multi-modal Degradation Feature Learning for Unified Image Restoration Based on Contrastive Learning

Learning Spatially Variant Degradation for Unsupervised Blind Photoacoustic Tomography Image Restoration

Learning Distortion Invariant Representation for Image Restoration from A Causality Perspective

Perceptual Image Restoration with High-Quality Priori and Degradation Learning

Training-Free Large Model Priors for Multiple-in-One Image Restoration

Boosting denoisers with reinforcement learning for image restoration

Hybrid sparsity learning for image restoration: An iterative and trainable approach

Ua-Dip: Unsupervised Image Deblurring Via Admm and Deep Image Prior

Plug-and-Play Image Restoration with Deep Denoiser Prior

Analysis of Deep Image Prior and Exploiting Self-Guidance for Image Reconstruction

Taming Generative Diffusion Prior for Universal Blind Image Restoration

Blind multi-Poissonian image deconvolution with sparse log-step gradient prior

Learning the Hybrid Nonlocal Self-Similarity Prior for Image Restoration

Priors in Deep Image Restoration and Enhancement: A Survey