Abstract:Recent generative-prior-based methods have shown promising blind face restoration performance. They usually project the degraded images to the latent space and then decode high-quality faces either by single-stage latent optimization or directly from the encoding. Generating fine-grained facial details faithful to inputs remains a challenging problem. Most existing methods produce either overly smooth outputs or alter the identity as they attempt to balance between generation and reconstruction. This may be attributed to the typical trade-off between quality and resolution in the latent space. If the latent space is highly compressed, the decoded output is more robust to degradations but shows worse fidelity. On the other hand, a more flexible latent space can capture intricate facial details better, but is extremely difficult to optimize for highly degraded faces using existing techniques. To address these issues, we introduce a diffusion-based-prior inside a VQGAN architecture that focuses on learning the distribution over uncorrupted latent embeddings. With such knowledge, we iteratively recover the clean embedding conditioning on the degraded counterpart. Furthermore, to ensure the reverse diffusion trajectory does not deviate from the underlying identity, we train a separate Identity Recovery Network and use its output to constrain the reverse diffusion process. Specifically, using a learnable latent mask, we add gradients from a face-recognition network to a subset of latent features that correlates with the finer identity-related details in the pixel space, leaving the other features untouched. Disentanglement between perception and fidelity in the latent space allows us to achieve the best of both worlds. We perform extensive evaluations on multiple real and synthetic datasets to validate the superiority of our approach.

VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decoder

Degradation-Aware Blind Face Restoration Via High-Quality VQ Codebook.

Virtual Completion of Facial Image in Ancient Murals

LD-BFR: Vector-Quantization-Based Face Restoration Model with Latent Diffusion Enhancement

Efficient Video Face Enhancement with Enhanced Spatial-Temporal Consistency

CLR-Face: Conditional Latent Refinement for Blind Face Restoration Using Score-Based Diffusion Models

VQ-NeRF: Vector Quantization Enhances Implicit Neural Representations

SVFR: A Unified Framework for Generalized Video Face Restoration

BFRFormer: Transformer-based generator for Real-World Blind Face Restoration

Blind Face Restoration via Deep Multi-scale Component Dictionaries

RestoreFormer: High-Quality Blind Face Restoration from Undegraded Key-Value Pairs

Blind Face Restoration Via Integrating Face Shape and Generative Priors

WaveFace: Authentic Face Restoration with Efficient Frequency Recovery

Blind Face Restoration Via Multi-Prior Collaboration and Adaptive Feature Fusion.

Exploring Correlations in Degraded Spatial Identity Features for Blind Face Restoration

TFRGAN: Leveraging Text Information for Blind Face Restoration with Extreme Degradation.

Towards Real-World Blind Face Restoration with Generative Facial Prior

Survey on Deep Face Restoration: From Non-blind to Blind and Beyond

Facial Landmarks and Generative Priors Guided Blind Face Restoration

AuthFace: Towards Authentic Blind Face Restoration with Face-oriented Generative Diffusion Prior

Visual Style Prompt Learning Using Diffusion Models for Blind Face Restoration