Abstract:Blind face restoration endeavors to restore a clear face image from a degraded counterpart. Recent approaches employing Generative Adversarial Networks (GANs) as priors have demonstrated remarkable success in this field. However, these methods encounter challenges in achieving a balance between realism and fidelity, particularly in complex degradation scenarios. To inherit the exceptional realism generative ability of the diffusion model and also constrained by the identity-aware fidelity, we propose a novel diffusion-based framework by embedding the 3D facial priors as structure and identity constraints into a denoising diffusion process. Specifically, in order to obtain more accurate 3D prior representations, the 3D facial image is reconstructed by a 3D Morphable Model (3DMM) using an initial restored face image that has been processed by a pretrained restoration network. A customized multi-level feature extraction method is employed to exploit both structural and identity information of 3D facial images, which are then mapped into the noise estimation process. In order to enhance the fusion of identity information into the noise estimation, we propose a Time-Aware Fusion Block (TAFB). This module offers a more efficient and adaptive fusion of weights for denoising, considering the dynamic nature of the denoising process in the diffusion model, which involves initial structure refinement followed by texture detail enhancement. Extensive experiments demonstrate that our network performs favorably against state-of-the-art algorithms on synthetic and real-world datasets for blind face restoration. The Code is released on our project page at <a class="link-external link-https" href="https://github.com/838143396/3Diffusion" rel="external noopener nofollow">this https URL</a>.

Real-World Blind Face Restoration with Generative Facial Prior and Degradation Simulation

Towards Real-World Blind Face Restoration with Generative Facial Prior

Enhancing Quality of Pose-varied Face Restoration with Local Weak Feature Sensing and GAN Prior

Feature-Guided Blind Face Restoration with GAN Prior

TFRGAN: Leveraging Text Information for Blind Face Restoration with Extreme Degradation.

Blind Face Restoration Via Integrating Face Shape and Generative Priors

Toward Real-World Blind Face Restoration With Generative Diffusion Prior

Towards Real-World Blind Face Restoration with Generative Diffusion Prior

Degradation learning and Skip-Transformer for blind face restoration

Blind Face Video Restoration with Temporal Consistent Generative Prior and Degradation-Aware Prompt

Facial Landmarks and Generative Priors Guided Blind Face Restoration

BFRFormer: Transformer-based generator for Real-World Blind Face Restoration

GAN Prior Embedded Network for Blind Face Restoration in the Wild

AuthFace: Towards Authentic Blind Face Restoration with Face-oriented Generative Diffusion Prior

DEAR-GAN: Degradation-Aware Face Restoration with GAN Prior

Blind Face Restoration Via Multi-Prior Collaboration and Adaptive Feature Fusion.

HPG-GAN: High-Quality Prior-Guided Blind Face Restoration Generative Adversarial Network

Learning Warped Guidance for Blind Face Restoration

3D Priors-Guided Diffusion for Blind Face Restoration

Blind Face Restoration under Extreme Conditions: Leveraging 3D-2D Prior Fusion for Superior Structural and Texture Recovery

Enhanced Blind Face Restoration With Multi-Exemplar Images and Adaptive Spatial Feature Fusion