Abstract:Blind face restoration is a challenging task due to the unknown, unsynthesizable and complex degradation, yet is valuable in many practical applications. To improve the performance of blind face restoration, recent works mainly treat the two aspects, i.e., generic and specific restoration, separately. In particular, generic restoration attempts to restore the results through general facial structure prior, while on the one hand, cannot generalize to real-world degraded observations due to the limited capability of direct CNNs' mappings in learning blind restoration, and on the other hand, fails to exploit the identity-specific details. On the contrary, specific restoration aims to incorporate the identity features from the reference of the same identity, in which the requirement of proper reference severely limits the application scenarios. Generally, it is a challenging and intractable task to improve the photo-realistic performance of blind restoration and adaptively handle the generic and specific restoration scenarios with a single unified model. Instead of implicitly learning the mapping from a low-quality image to its high-quality counterpart, this paper suggests a DMDNet by explicitly memorizing the generic and specific features through dual dictionaries. First, the generic dictionary learns the general facial priors from high-quality images of any identity, while the specific dictionary stores the identity-belonging features for each person individually. Second, to handle the degraded input with or without specific reference, dictionary transform module is suggested to read the relevant details from the dual dictionaries which are subsequently fused into the input features. Finally, multi-scale dictionaries are leveraged to benefit the coarse-to-fine restoration. The whole framework including the generic and specific dictionaries is optimized in an end-to-end manner and c- n be flexibly plugged into different application scenarios. Moreover, a new high-quality dataset, termed CelebRef-HQ, is constructed to promote the exploration of specific face restoration in the high-resolution space. Experimental results demonstrate that the proposed DMDNet performs favorably against the state of the arts in both quantitative and qualitative evaluation, and generates more photo-realistic results on the real-world low-quality images. The codes, models and the CelebRef-HQ dataset will be publicly available at https://github.com/csxmli2016/DMDNet.

Multiprior Learning via Neural Architecture Search for Blind Face Restoration

Fast non-iterative blind restoration of hyperspectral images with PSFs

Enhancing Quality of Pose-varied Face Restoration with Local Weak Feature Sensing and GAN Prior

AuthFace: Towards Authentic Blind Face Restoration with Face-oriented Generative Diffusion Prior

Blind Face Restoration under Extreme Conditions: Leveraging 3D-2D Prior Fusion for Superior Structural and Texture Recovery

BFRFormer: Transformer-based generator for Real-World Blind Face Restoration

Face Restoration via Plug-and-Play 3D Facial Priors

Blind Face Restoration Via Integrating Face Shape and Generative Priors

Survey on Deep Face Restoration: From Non-blind to Blind and Beyond

Towards Real-World Blind Face Restoration with Generative Diffusion Prior

Toward Real-World Blind Face Restoration With Generative Diffusion Prior

GAN Prior Embedded Network for Blind Face Restoration in the Wild

A Survey of Deep Face Restoration: Denoise, Super-Resolution, Deblur, Artifact Removal

Blind Face Restoration via Deep Multi-scale Component Dictionaries

A Double Feature Fusion Network with Progressive Learning for Sharper Inpainting

3D Priors-Guided Diffusion for Blind Face Restoration

Deep Face Super-Resolution with Iterative Collaboration between Attentive Recovery and Landmark Estimation

Learning Dual Memory Dictionaries for Blind Face Restoration

RestoreFormer: High-Quality Blind Face Restoration from Undegraded Key-Value Pairs

Facial mask attention network for identity-aware face super-resolution

HPG-GAN: High-Quality Prior-Guided Blind Face Restoration Generative Adversarial Network