BFRFormer: Transformer-based generator for Real-World Blind Face Restoration

Guojing Ge,Qi Song,Guibo Zhu,Yuting Zhang,Jinglu Chen,Miao Xin,Ming Tang,Jinqiao Wang

2024-02-29

Abstract:Blind face restoration is a challenging task due to the unknown and complex degradation. Although face prior-based methods and reference-based methods have recently demonstrated high-quality results, the restored images tend to contain over-smoothed results and lose identity-preserved details when the degradation is severe. It is observed that this is attributed to short-range dependencies, the intrinsic limitation of convolutional neural networks. To model long-range dependencies, we propose a Transformer-based blind face restoration method, named BFRFormer, to reconstruct images with more identity-preserved details in an end-to-end manner. In BFRFormer, to remove blocking artifacts, the wavelet discriminator and aggregated attention module are developed, and spectral normalization and balanced consistency regulation are adaptively applied to address the training instability and over-fitting problem, respectively. Extensive experiments show that our method outperforms state-of-the-art methods on a synthetic dataset and four real-world datasets. The source code, Casia-Test dataset, and pre-trained models are released at

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the challenges in **Blind Face Restoration (BFR)**, that is, to restore high - quality facial images from low - quality facial images, and these low - quality images may suffer from unknown and complex degradations. Specifically, existing methods often produce over - smoothed results when dealing with severely degraded images and lose identity - preserving details.

### Specific manifestations of the problem
1. **Over - smoothing**: Existing methods are prone to generate over - smoothed results when dealing with severely degraded images, resulting in the loss of details.
2. **Loss of identity information**: When the degradation is severe, it is difficult for existing methods to preserve the identity characteristics of the original image.
3. **Short - range dependence**: The inherent limitations of Convolutional Neural Networks (CNN) make it difficult for them to model long - range dependence, thus affecting the restoration effect.

### Solutions proposed in the paper
To solve the above problems, the author proposes a Transformer - based blind face restoration method, named **BFRFormer**. The main innovations of this method include:
- **Transformer architecture**: Use Transformer to model long - range dependence, thereby improving the over - smoothing problem and retaining more identity details.
- **Wavelet Discriminator**: Used to remove blocky artifacts and generate more realistic facial details.
- **Aggregated Attention Module (AAM)**: Combine channel attention and dual - attention mechanisms to expand the effective receptive field and activate more input pixels.
- **Spectral Normalization and Balanced Consistency Regulation (bCR)**: Used to solve training instability and over - fitting problems respectively.

### Main contributions
1. Proposed a Transformer - based blind face restoration method, embedded in the GAN prior framework and trained in an end - to - end manner.
2. Designed a new Aggregated Attention Module (AAM), which combines global and local information to improve the restoration effect.
3. Constructed a new real - world test data set, which contains more extensive diversity, such as different races, ages, occlusions, etc.
4. Experimental results show that this method outperforms existing methods on both synthetic data sets and four real - world data sets.

Through these improvements, BFRFormer can better preserve identity information and generate high - quality facial details when dealing with severely degraded images.