Abstract:Super-resolution (SR) techniques aim to restore a high-resolution (HR) image from low-resolution (LR) images, which are often used to assist the enhancement of image/video quality under the rapid development of HR and high-frame-rate media. Recently, neural network (NN)-based methods perform much better image reconstruction quality than classical approaches. However, the unacceptable computation complexity as well as the huge memory footprints of NNs limit the throughputs and scalability of these SR systems. In this work, we analyze several key issues in the design of NN-based SR systems first. Then, we propose a three-level systematic optimization methodology for SR systems to reduce computation overhead and keep image quality. At the algorithm level, we introduce image blocking to SR tasks and develop a block-wise SR algorithm based on the hybrid of NN and interpolation with a consistent image block evaluation metric. The configurable hybrid parameters help the SR algorithm to achieve a flexible tradeoff between the computation overhead and image quality. At the operator level, we focus on the transpose convolution operators commonly used for upsampling in SR NNs. We propose an efficient Winograd-based transposed convolution acceleration method. Through the efficient subconvolutions conversion and the Winograd specialization, this methods enables unified Winograd transformations and simplified data access patterns. At the data level, we propose a novel quantization method for Winograd-aware SR NNs to get better-quantized accuracy. Comprehensive evaluations demonstrate the effectiveness of these optimizations. Our SR system reduces a large number of multiplications with great scalability and supports 4K@120 fps and 8K@30 fps outputs with acceptable image quality degradation.

ADAPTIVE IMAGE SUPER-RESOLUTION ALGORITHM BASED ON FRACTIONAL FOURIER TRANSFORM

Fast Image Super-Resolution for a Dual-Resolution Camera

Super-resolution Reconstruction Algorithms Based on Fusion of Deep Learning Mechanism and Wavelet

Single Image Super Resolution Reconstruction Algorithm Based on Deep Learning

FSR: A General Frequency-Oriented Framework to Accelerate Image Super-resolution Networks.

Frequency Reciprocal Action and Fusion for Single Image Super-Resolution

Single image super-resolution using self-optimizing mask via fractional-order gradient interpolation and reconstruction

Multi-Frame Super Resolution With Deep Residual Learning On Flow Registered Non-Integer Pixel Images

A Practical And Adaptive Framework For Super-Resolution

Spatially Adaptive Block-Based Super-Resolution

Image Super-Resolution Reconstruction Model Based on Multi-Feature Fusion

Deep Fourier-based Arbitrary-scale Super-resolution for Real-time Rendering

Infrared image super-resolution reconstruction based on residual fast fourier transform

Faster Image Super-Resolution by Improved Frequency-Domain Neural Networks.

Efficient Super-Resolution System with Block-Wise Hybridization and Quantized Winograd on FPGA

Adaptive Fusion Super-Resolution Based on Parallel Scoring

Single image super resolution via wavelet transform fusion and SRFeat network

Efficient Adaptive Feature Fusion Network for Remote-Sensing Image Super-Resolution

Single Image Super Resolution Enhancement Method Based on Fractal Interpolation of Iteration Function System

Iterative-in-Iterative Super-Resolution Biomedical Imaging Using One Real Image

Interpolation-based Image Super-Resolution Using Multisurface Fitting.