Abstract:Super-resolution (SR) techniques aim to restore a high-resolution (HR) image from low-resolution (LR) images, which are often used to assist the enhancement of image/video quality under the rapid development of HR and high-frame-rate media. Recently, neural network (NN)-based methods perform much better image reconstruction quality than classical approaches. However, the unacceptable computation complexity as well as the huge memory footprints of NNs limit the throughputs and scalability of these SR systems. In this work, we analyze several key issues in the design of NN-based SR systems first. Then, we propose a three-level systematic optimization methodology for SR systems to reduce computation overhead and keep image quality. At the algorithm level, we introduce image blocking to SR tasks and develop a block-wise SR algorithm based on the hybrid of NN and interpolation with a consistent image block evaluation metric. The configurable hybrid parameters help the SR algorithm to achieve a flexible tradeoff between the computation overhead and image quality. At the operator level, we focus on the transpose convolution operators commonly used for upsampling in SR NNs. We propose an efficient Winograd-based transposed convolution acceleration method. Through the efficient subconvolutions conversion and the Winograd specialization, this methods enables unified Winograd transformations and simplified data access patterns. At the data level, we propose a novel quantization method for Winograd-aware SR NNs to get better-quantized accuracy. Comprehensive evaluations demonstrate the effectiveness of these optimizations. Our SR system reduces a large number of multiplications with great scalability and supports 4K@120 fps and 8K@30 fps outputs with acceptable image quality degradation.

Interactive super-resolution through neighbor embedding

Epistemic-Uncertainty-Based Divide-and-Conquer Network for Single-Image Super-Resolution

Image Super-Resolution with Sparse Neighbor Embedding

A Super-resolution Algorithm Through Neighbor Embedding

Neural Super-Resolution in Real-Time Rendering Using Auxiliary Feature Enhancement

Single Remote Sensing Image Super-Resolution Via a Generative Adversarial Network with Stratified Dense Sampling and Chain Training

Efficient Deep Models for Real-Time 4K Image Super-Resolution. NTIRE 2023 Benchmark and Report.

Partially Supervised Neighbor Embedding for Example-Based Image Super-Resolution

Efficient Super-Resolution System with Block-Wise Hybridization and Quantized Winograd on FPGA

Neighbor embedding based super-resolution algorithm through edge detection and feature selection

Image Processing GNN: Breaking Rigidity in Super-Resolution

From Coarse to Fine: Hierarchical Pixel Integration for Lightweight Image Super-resolution

Edge-Preserving Single Remote-Sensing Image Super-resolution Using Sparse Representations

A Fast And Accurate Super-Resolution Network Using Progressive Residual Learning

Joint learning for single-image super-resolution via a coupled constraint.

Super-Resolution by Predicting Offsets: An Ultra-Efficient Super-Resolution Network for Rasterized Images.

Multi-level feature interaction image super-resolution network based on convolutional nonlinear spiking neural model

Single Image Super-Resolution Reconstruction of Enhanced Loss Function with Multi-GPU Training

Efficient Hybrid Feature Interaction Network for Stereo Image Super-Resolution

Image Super-Resolution via Hierarchical and Collaborative Sparse Representation

Improved Super-Resolution Through Residual Neighbor Embedding