Abstract:Deep neural networks have achieved remarkable success in single image super-resolution (SISR). The computing and memory requirements of these methods have hindered their application to broad classes of real devices with limited computing power, however. One approach to this problem has been lightweight network architectures that balance the super-resolution performance and the computation burden. In this study, we revisit this problem from an orthogonal view, and propose a novel learning strategy to maximize the pixel-wise fitting ability of a given lightweight network architecture. Considering that the initial performance of the lightweight network is very limited, we present an adaptive importance learning scheme for SISR that trains the network with an easy-to-complex paradigm by dynamically updating the importance of image pixels on the basis of the training loss. Specifically, we formulate the network training and the importance learning into a joint optimization problem. With a carefully designed importance penalty function, the importance of individual pixels can be gradually increased through solving a convex optimization problem. The training process thus begins with pixels that are easy to reconstruct, and gradually proceeds to more complex pixels as fitting improves. Furthermore, the proposed learning scheme is able to seamlessly assimilate knowledge from a more powerful teacher network in the form of importance initialization, thus obtaining better initial performance for the network. Through learning the network parameters, and updating pixel importance, the proposed learning scheme enables smaller, lightweight, networks to achieve better performance than has previously been possible. Extensive experiments on four benchmark datasets demonstrate the potential benefits of the proposed learning strategy in lightweight SISR network enhancement. In some cases, our learned network with only 25 % of the parameters and computational complexity can produce comparable or even better results than the corresponding full-parameter network.

Two Strategies Toward Lightweight Image Super-Resolution

Lightweight Image Super-Resolution Network Using 3D Convolutional Neural Networks

Lightweight Multi-Attention Fusion Network for Image Super-Resolution

Lightweight Attention-Guided Network for Image Super-Resolution

Adaptive Loss Function for Super Resolution Neural Networks Using Convex Optimization Techniques

Lightweight multi-scale residual networks with attention for image super-resolution

Lightweight global-locally connected distillation network for single image super-resolution

Lightweight adaptive weighted network for single image super-resolution

Fast and Accurate Single Image Super-Resolution Via an Energy-Aware Improved Deep Residual Network.

Lightweight image super-resolution with enhanced CNN

Lightweight Image Super-Resolution with Adaptive Weighted Learning Network

Single Image Super-resolution via a Lightweight Residual Convolutional Neural Network

A lightweight network with bidirectional constraints for single image super-resolution

Lightweight Image Super-resolution with Local Attention Enhancement

A Lightweight Image Super-Resolution Reconstruction Algorithm Based on the Residual Feature Distillation Mechanism

Spatial and Channel Aggregation Network for Lightweight Image Super-Resolution

A very lightweight and efficient image super-resolution network

Adaptive Importance Learning for Improving Lightweight Image Super-resolution Network.

Lightweight single image super-resolution based on multi-path progressive feature fusion and attention mechanism

A Lightweight Multi-Scale Channel Attention Network for Image Super-Resolution.

Large Factor Image Super-Resolution with Cascaded Convolutional Neural Networks.