Abstract:Recently, transformer-based backbones show superior performance over the convolutional counterparts in computer vision. Due to quadratic complexity with respect to the token number in global attention, local attention is always adopted in low-level image processing with linear complexity. However, the limited receptive field is harmful to the performance. In this paper, motivated by Octave convolution, we propose a transformer-based single image super-resolution (SISR) model, which explicitly embeds dynamic frequency decomposition into the standard local transformer. All the frequency components are continuously updated and re-assigned via intra-scale attention and inter-scale interaction, respectively. Specifically, the attention in low resolution is enough for low-frequency features, which not only increases the receptive field, but also decreases the complexity. Compared with the standard local transformer, the proposed FDRTran layer simultaneously decreases FLOPs and parameters. By contrast, Octave convolution only decreases FLOPs of the standard convolution, but keeps the parameter number unchanged. In addition, the restart mechanism is proposed for every a few frequency updates, which first fuses the low and high frequency, then decomposes the features again. In this way, the features can be decomposed in multiple viewpoints by learnable parameters, which avoids the risk of early saturation for frequency representation. Furthermore, based on the FDRTran layer with restart mechanism, the proposed FDRNet is the first transformer backbone for SISR which discusses the Octave design. Sufficient experiments show our model reaches state-of-the-art performance on 6 synthetic and real datasets. The code and the models are available at https://github.com/catnip1029/FDRNet.

Cascade Residual-Optimized Image Super-Resolution Reconstruction in Transformer Network

Super-resolution Reconstruction Algorithms Based on Fusion of Deep Learning Mechanism and Wavelet

Multi-Scale Cross-Attention Fusion Network Based on Image Super-Resolution

A Single Super-Resolution Method Via Deep Cascade Network

The super-resolution reconstruction algorithm of multi-scale dilated convolution residual network

DTCNet: Transformer-CNN Distillation for Super-Resolution of Remote Sensing Image

Cascaded Deep Networks with Multiple Receptive Fields for Infrared Image Super-Resolution

An Efficient Hybrid CNN-Transformer Approach for Remote Sensing Super-Resolution

Hybrid-Scale Hierarchical Transformer for Remote Sensing Image Super-Resolution

A Residual Network with Efficient Transformer for Lightweight Image Super-Resolution

Super-Resolution Algorithm Based on Transformer+CNN

Image Super-resolution via Efficient Transformer Embedding Frequency Decomposition with Restart

Cross-Spatial Pixel Integration and Cross-Stage Feature Fusion Based Transformer Network for Remote Sensing Image Super-Resolution

Image Super-resolution Reconstruction Network based on Enhanced Swin Transformer via Alternating Aggregation of Local-Global Features

Image Super-Resolution Reconstruction Based on Dense Residual Attention and Multi-Scale Feature Fusion

An efficient parallel fusion structure of distilled and transformer-enhanced modules for lightweight image super-resolution

DRCT: Saving Image Super-resolution away from Information Bottleneck

MadFormer: multi-attention-driven image super-resolution method based on Transformer

Research on Image Super-Resolution Reconstruction Mechanism based on Convolutional Neural Network

The Method of Industrial Internet Image Super-resolution Based on Transformer

Single image super-resolution with multi-scale information cross-fusion network