Abstract:Recently, deep learning technique has been widely employed to deal with face super-resolution (FSR) problem. It aims to predict the nonlinear relationship between the low-resolution (LR) face images and corresponding high-resolution (HR) ones, which could recover the high-frequency details from the LR degraded textures. However, either CNN-based or Transformer-based approaches mostly enhance the details by exploiting the relationship of local pixels or patches on LR features, the nonlocal features are not fully taken into account for producing high-frequency textures. To improve the above problem, we design a novel dual-branch module which consists of Transformer and CNN respectively. The Transformer branch extracts multiple scale feature embeddings and explores local and nonlocal self-attention simultaneously. Thus, the parallel self-attention mechanism has superior capabilities to capture the local and nonlocal dependencies on face image in the face reconstruction. Furthermore, the traditional CNNs usually extract features by combining pixels in a local convolutional kernel, it may be not effective to recover lost high-frequency details since the variations of local pixels are not well measured, which is important in recovering vivid edges and contours. To this end, we propose the local variation based attention block on the CNN branch, which could enhance the capabilities by directly extracting features from the variation of neighboring pixels. Finally, the Transformer-branch and CNN-branch are combined together by the modulation block to fuse both nonlocal and local advantages from two branches. Experimental results demonstrate the effectiveness of the proposed method when compared with state-of-the-art approaches. The source code is available https://github.com/jingang-cv/DBTC.

Multi-Scale Feature Fusion and Structure-Preserving Network for Face Super-Resolution

Multi-Scale Cross-Attention Fusion Network Based on Image Super-Resolution

Face Super-Resolution Via Progressive-Scale Boosting Network

Multi-Scale Spatial Transformation and Attention Based Super-Resolution Reconstruction Algorithm

Face Super-Resolution Via Hierarchical Multi-Scale Residual Fusion Network.

A Face Structure Attention Network for Face Super-Resolution

The face image super-resolution algorithm based on combined representation learning

Single-image Super-Resolution Via Selective Multi-Scale Network

Compressed Multi-Scale Feature Fusion Network for Single Image Super-Resolution.

Multi-Scale Residual Network For Image Super-Resolution

Image super-resolution reconstruction based on attention mechanism and feature fusion

Face Super-Resolution with Progressive Embedding of Multi-scale Face Priors

An Efficient Multi-scale Method for Single Image Super Resolution.

A Multi-Scale Recursive Attention Feature Fusion Network for Image Super-Resolution Reconstruction Algorithm

Noise face image super-resolution reconstruction based on multi-information fusion

Exploiting Multi-scale Parallel Self-attention and Local Variation via Dual-branch Transformer-CNN Structure for Face Super-resolution

Image Super-Resolution Reconstruction Based on Dense Residual Attention and Multi-Scale Feature Fusion

Fused Pyramid Attention Network for Single Image Super-Resolution.

Attention-Guided Multi-scale Interaction Network for Face Super-Resolution

Learning Face Image Super-Resolution Through Facial Semantic Attribute Transformation and Self-Attentive Structure Enhancement

Single image super-resolution with multi-scale information cross-fusion network