Abstract:Recent generative-prior-based methods have shown promising blind face restoration performance. They usually project the degraded images to the latent space and then decode high-quality faces either by single-stage latent optimization or directly from the encoding. Generating fine-grained facial details faithful to inputs remains a challenging problem. Most existing methods produce either overly smooth outputs or alter the identity as they attempt to balance between generation and reconstruction. This may be attributed to the typical trade-off between quality and resolution in the latent space. If the latent space is highly compressed, the decoded output is more robust to degradations but shows worse fidelity. On the other hand, a more flexible latent space can capture intricate facial details better, but is extremely difficult to optimize for highly degraded faces using existing techniques. To address these issues, we introduce a diffusion-based-prior inside a VQGAN architecture that focuses on learning the distribution over uncorrupted latent embeddings. With such knowledge, we iteratively recover the clean embedding conditioning on the degraded counterpart. Furthermore, to ensure the reverse diffusion trajectory does not deviate from the underlying identity, we train a separate Identity Recovery Network and use its output to constrain the reverse diffusion process. Specifically, using a learnable latent mask, we add gradients from a face-recognition network to a subset of latent features that correlates with the finer identity-related details in the pixel space, leaving the other features untouched. Disentanglement between perception and fidelity in the latent space allows us to achieve the best of both worlds. We perform extensive evaluations on multiple real and synthetic datasets to validate the superiority of our approach.

Visual Style Prompt Learning Using Diffusion Models for Blind Face Restoration

3D Priors-Guided Diffusion for Blind Face Restoration

ArtBank: Artistic Style Transfer with Pre-trained Diffusion Model and Implicit Style Prompt Bank

Towards Unsupervised Blind Face Restoration using Diffusion Prior

Towards Real-World Blind Face Restoration with Generative Diffusion Prior

Toward Real-World Blind Face Restoration With Generative Diffusion Prior

OSDFace: One-Step Diffusion Model for Face Restoration

AuthFace: Towards Authentic Blind Face Restoration with Face-oriented Generative Diffusion Prior

Blind Face Video Restoration with Temporal Consistent Generative Prior and Degradation-Aware Prompt

Enhanced Blind Face Restoration with Multi-Exemplar Images and Adaptive Spatial Feature Fusion

CLR-Face: Conditional Latent Refinement for Blind Face Restoration Using Score-Based Diffusion Models

Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model

Blind Face Restoration Via Integrating Face Shape and Generative Priors

Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restoration

DifFace: Blind Face Restoration with Diffused Error Contraction

Degradation learning and Skip-Transformer for blind face restoration

Enhancing Quality of Pose-varied Face Restoration with Local Weak Feature Sensing and GAN Prior

Implicit Subspace Prior Learning for Dual-Blind Face Restoration

Survey on Deep Face Restoration: From Non-blind to Blind and Beyond

Textual Prompt Guided Image Restoration