Abstract:In this paper, we address the unified image restoration challenge by reframing it as a contrastive learning-based classification problem. Despite the significant strides made by deep learning methods in enhancing image restoration quality, their limited capacity to generalize across diverse degradation types and intensities necessitates the training of separate models for each specific degradation scenario. We proposes an all-encompassing approach that can restore images from various unknown corruption types and levels. We devise a method that learns representations of the latent sharp image’s degradation and accompanying textual features (such as dataset categories and image content descriptions), converting these into prompts which are then embedded within a reconstruction network model to enhance cross-database restoration performance. This culminates in a unified image reconstruction framework. The study involves two stages: In the first stage, we design a MultiContentNet that learns multi-modal features (MMFs) of the latent sharp image. This network encodes the visual degradation expressions and contextual text features into latent variables, thereby exerting a guided classification effect. Specifically, MultiContentNet is trained as an auxiliary controller capable of taking the degraded input image and, through contrastive learning, extracts MMFs of the latent target image. This effectively generates natural classifiers tailored for different degradation types. The second phase integrates the learned MMFs into an image restoration network via cross-attention mechanisms. This guides the restoration model to learn high-fidelity image recovery. Experiments conducted on six blind image restoration tasks demonstrate that the proposed method achieves state-of-the-art performance, highlighting the potential significance of large-scale pretrained vision-language models’ MMFs in advancing high-quality unified image reconstruction.

Improving Visual Quality and Transferability of Adversarial Attacks on Face Recognition Simultaneously with Adversarial Restoration

DifFace: Blind Face Restoration with Diffused Error Contraction

RobustFace: a novel image restoration technique for face adversarial robustness improvement

A Double Feature Fusion Network with Progressive Learning for Sharper Inpainting

Multi-modal Degradation Feature Learning for Unified Image Restoration Based on Contrastive Learning

Improving the Transferability of Adversarial Attacks on Face Recognition With Beneficial Perturbation Feature Augmentation

AuthFace: Towards Authentic Blind Face Restoration with Face-oriented Generative Diffusion Prior

Enhancing the Transferability of Adversarial Attacks on Face Recognition with Diverse Parameters Augmentation

RestoreFormer: High-Quality Blind Face Restoration from Undegraded Key-Value Pairs

Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model

Towards Authentic Face Restoration with Iterative Diffusion Models and Beyond

Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Model

An Improved Face Image Restoration Method Based on Denoising Diffusion Probabilistic Models

Real-World Blind Face Restoration with Generative Facial Prior and Degradation Simulation

Toward Transferable Attack via Adversarial Diffusion in Face Recognition

Adversarial Purification and Fine-tuning for Robust UDC Image Restoration

3D Priors-Guided Diffusion for Blind Face Restoration

DiffLoss: unleashing diffusion model as constraint for training image restoration network

Learning to Restore 3D Face from In-the-Wild Degraded Images

Towards Unsupervised Blind Face Restoration using Diffusion Prior

NeRFTAP: Enhancing Transferability of Adversarial Patches on Face Recognition Using Neural Radiance Fields