Abstract:Current super-resolution networks typically reduce network parameters and multiadds operations by designing lightweight structures, but lightening the convolution layer is often ignored. In this work, we observe that $3 \times 3$ convolutions occupy a high percentage of network parameters in most lightweight super-resolution networks. This motivates us to consider lightening super-resolution networks by replacing $3 \times 3$ convolutions with lightweight convolutions, while maintaining the performance. To achieve this, we propose a lightweight convolution layer named contextual transformation layer (CTL). It can yield efficient contextual features through a context feature extraction module and enrich extracted contextual features through a context feature transformation module. Based on CTLs, we build a lightweight super-resolution network called contextual transformation network (CTN) for remote-sensing image super-resolution. Specifically, we use two CTLs to construct a contextual transformation block (CTB) for hierarchical feature learning. Interleaved with a CTB, a context enhancement module (CEM) is employed to enhance the extracted feature representations. All extracted features are processed by a contextual feature aggregation module for final remote-sensing image super-resolution. Extensive experiments are performed on a remote-sensing image super-resolution benchmark named UC Merced. Our method achieves superior results to the other state-of-the-art methods. To demonstrate the generalization ability of our CTL, we extend our CTN to two relevant tasks: natural image super-resolution and natural image denoising. Experimental results on natural image super-resolution benchmarks (i.e., Set5, Set14, B100, Urban100, and Manga109) and natural image denoising benchmarks (i.e., SIDD and DND) further prove the superiority of our method. Our code is publicly available at https://github.com/BITszwang/CTNet.

Contextual Convolutional Networks

Contextual Convolutional Neural Networks

Spatially-Aware Context Neural Networks.

Contextual Transformer Networks for Visual Recognition

Deep Context-Aware Kernel Networks

Convolutional Networks with Cross-Layer Neurons for Image Recognition

A ConvNet for the 2020s

Connectivity-Inspired Network for Context-Aware Recognition

2 Related Work Following the success of deep neural networks for image classification

Weakly-Supervised Spatial Context Networks.

C-volution: A Hybrid Operator for Visual Recognition.

A Simple Convolutional Transfer Neural Networks In Vision Tasks

Learning Contextual Dependencies with Convolutional Hierarchical Recurrent Neural Networks

CG-CNN: Self-Supervised Feature Extraction Through Contextual Guidance and Transfer Learning

Deep Multi-Context Network for FINE-GRAINED VISUAL RECOGNITION

Contextual Transformation Network for Lightweight Remote-Sensing Image Super-Resolution

Learning Contextual Transformer Network for Image Inpainting

Deep Context Convolutional Neural Networks for Semantic Segmentation.

Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition

Contextual associations represented both in neural networks and human behavior