Abstract:Traditional Convolutional Neural Network (ConvNet, CNN)-based image super-resolution (SR) methods have lower computation costs, making them more friendly for real-world scenarios. However, they suffer from lower performance. On the contrary, Vision Transformer (ViT)-based SR methods have achieved impressive performance recently, but these methods often suffer from high computation costs and model storage overhead, making them hard to meet the requirements in practical application scenarios. In practical scenarios, an SR model should reconstruct an image with high quality and fast inference. To handle this issue, we propose a novel CNN-based Efficient Residual ConvNet enhanced with structural Re-parameterization (RepECN) for a better trade-off between performance and efficiency. A stage-to-block hierarchical architecture design paradigm inspired by ViT is utilized to keep the state-of-the-art performance, while the efficiency is ensured by abandoning the time-consuming Multi-Head Self-Attention (MHSA) and by re-designing the block-level modules based on CNN. Specifically, RepECN consists of three structural modules: a shallow feature extraction module, a deep feature extraction, and an image reconstruction module. The deep feature extraction module comprises multiple ConvNet Stages (CNS), each containing 6 Re-Parameterization ConvNet Blocks (RepCNB), a head layer, and a residual connection. The RepCNB utilizes larger kernel convolutions rather than MHSA to enhance the capability of learning long-range dependence. In the image reconstruction module, an upsampling module consisting of nearest-neighbor interpolation and pixel attention is deployed to reduce parameters and maintain reconstruction performance, while bicubic interpolation on another branch allows the backbone network to focus on learning high-frequency information. The extensive experimental results on multiple public benchmarks show that our RepECN can achieve 2.5∼5× faster inference than the state-of-the-art ViT-based SR model with better or competitive super-resolving performance, indicating that our RepECN can reconstruct high-quality images with fast inference.

Generalized Enhanced Convolution Block for Super-Resolution Networks

Image Super-Resolution Using Residual Global Context Network.

Compact Cross-Reparam Convolution Network for Efficient Image Super-resolution.

RepSR: Training Efficient VGG-style Super-Resolution Networks with Structural Re-Parameterization and Batch Normalization

DR-Block: Convolutional Dense Reparameterization for CNN Generalization Free Improvement

Deformable and residual convolutional network for image super-resolution

Accurate and Efficient Image Super-Resolution via Global-Local Adjusting Dense Network

A Heterogeneous Group CNN for Image Super-Resolution

A New Convolutional Neural Network for Super-Resolution by Global and Local Residual

Efficient Self-Calibrated and Hierarchical Refinement Network for Lightweight Super-Resolution

Image Super-Resolution Via Residual Block Attention Networks.

CFGN: A Lightweight Context Feature Guided Network for Image Super-Resolution

RepECN: Making ConvNets Better Again for Efficient Image Super-Resolution

Lightweight Progressive Residual Clique Network for Image Super-Resolution.

Efficient Real-world Image Super-Resolution Via Adaptive Directional Gradient Convolution

A very lightweight and efficient image super-resolution network

Image Super-Resolution via Deep Feature Recalibration Network.

Cascading and Enhanced Residual Networks for Accurate Single-Image Super-Resolution.

Gradient-Guided and Multi-Scale Feature Network for Image Super-Resolution

Image Super-Resolution Based on Gated Residual and Gated Convolution Networks

Relation-consistency Graph Convolutional Network for Image Super-Resolution