Meta-USR: A Unified Super-Resolution Network for Multiple Degradation Parameters

Xuecai Hu,Zhang Zhang,Caifeng Shan,Zilei Wang,Liang Wang,Tieniu Tan

DOI: https://doi.org/10.1109/tnnls.2020.3016974

IF: 14.255

2021-09-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:Recent research on single image super-resolution (SISR) has achieved great success due to the development of deep convolutional neural networks. However, most existing SISR methods merely focus on super-resolution of a single fixed integer scale factor. This simplified assumption does not meet the complex conditions for real-world images which often suffer from various blur kernels or various levels of noise. More importantly, previous methods lack the ability to cope with arbitrary degradation parameters (scale factors, blur kernels, and noise levels) with a single model. A few methods can handle multiple degradation factors, e.g., noninteger scale factors, blurring, and noise, simultaneously within a single SISR model. In this work, we propose a simple yet powerful method termed meta-USR which is the first unified super-resolution network for arbitrary degradation parameters with meta-learning. In Meta-USR, a meta-restoration module (MRM) is proposed to enhance the traditional upscale module with the capability to adaptively predict the weights of the convolution filters for various combinations of degradation parameters. Thus, the MRM can not only upscale the feature maps with arbitrary scale factors but also restore the SR image with different blur kernels and noise levels. Moreover, the lightweight MRM can be placed at the end of the network, which makes it very efficient for iteratively/repeatedly searching the various degradation factors. We evaluate the proposed method through extensive experiments on several widely used benchmark data sets on SISR. The qualitative and quantitative experimental results show the superiority of our Meta-USR.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture

What problem does this paper attempt to address?

This paper introduces a new problem in solving Single Image Super-Resolution (SISR). Existing SISR methods mainly focus on image enlargement with fixed integer scaling factors. However, in reality, images may be affected by various blur kernels and different levels of noise. These existing methods cannot handle arbitrary degradation parameters (scaling factor, blur kernel, and noise level).

The paper proposes a unified super-resolution network called Meta-USR, which is the first to use meta-learning to handle arbitrary degradation parameters. Meta-USR consists of a Meta-Restoration Module (MRM) that enhances the traditional upsampling module and predicts the weights of convolution filters under different degradation parameter combinations. In this way, MRM can not only upscale feature maps with arbitrary scaling factors but also restore images with different blur kernels and noise levels.

Furthermore, due to the lightweight design of MRM, it can be placed at the end of the network, making it suitable for iterative computation and improving efficiency when dealing with complex degradation factors. By conducting numerous experiments on multiple widely used benchmark datasets, the paper demonstrates the superiority of the Meta-USR method.

In summary, the paper aims to address the problem of creating a single deep learning model that can adapt to various degradation conditions in real-world images, including arbitrary scaling factors, blur, and noise, in order to achieve efficient and flexible super-resolution reconstruction.