Abstract:Super-resolution (SR) techniques aim to restore a high-resolution (HR) image from low-resolution (LR) images, which are often used to assist the enhancement of image/video quality under the rapid development of HR and high-frame-rate media. Recently, neural network (NN)-based methods perform much better image reconstruction quality than classical approaches. However, the unacceptable computation complexity as well as the huge memory footprints of NNs limit the throughputs and scalability of these SR systems. In this work, we analyze several key issues in the design of NN-based SR systems first. Then, we propose a three-level systematic optimization methodology for SR systems to reduce computation overhead and keep image quality. At the algorithm level, we introduce image blocking to SR tasks and develop a block-wise SR algorithm based on the hybrid of NN and interpolation with a consistent image block evaluation metric. The configurable hybrid parameters help the SR algorithm to achieve a flexible tradeoff between the computation overhead and image quality. At the operator level, we focus on the transpose convolution operators commonly used for upsampling in SR NNs. We propose an efficient Winograd-based transposed convolution acceleration method. Through the efficient subconvolutions conversion and the Winograd specialization, this methods enables unified Winograd transformations and simplified data access patterns. At the data level, we propose a novel quantization method for Winograd-aware SR NNs to get better-quantized accuracy. Comprehensive evaluations demonstrate the effectiveness of these optimizations. Our SR system reduces a large number of multiplications with great scalability and supports 4K@120 fps and 8K@30 fps outputs with acceptable image quality degradation.

The High-speed Super-resolution Image Processing System

Design and Implementation of Visual Processing System Based on FPGA

High speed video signal acquisition and processing system based on FPGA technique

Fast Image Super-Resolution for a Dual-Resolution Camera

FPGA-Based Real-Time Super-Resolution System for Ultra High Definition Videos

Research of high speed image processing system based on FPGA

High Resolution X-Ray Medical Sequential Image Acquisition and Processing System Based on PCI Interface

Winograd-Based Real-Time Super-Resolution System on FPGA

FPGA Implementation of Non-Parametric Stereo Matching Algorithm

Design and Experiment of Prototype System for Parallel Processing of Remote Sensing Images Based on FPGA

FPGA Implementation of Feature Detection Algorithm Based on High Level Synthesis

A High-Performance Accelerator for Real-Time Super-Resolution on Edge FPGAs

FPGA implementation of full HD real-time depth estimation

Design of high-speed real-time sensor image processing based on FPGA and DDR3

A Real-Time Full HD 2D-to-3d Video Conversion System Based on FPGA

HDSuper: High-Quality and High Computational Utilization Edge Super-Resolution Accelerator With Hardware-Algorithm Co-Design Techniques

Efficient Super-Resolution System with Block-Wise Hybridization and Quantized Winograd on FPGA

Infrared Real-time Imaging System Based on DSP

High-performance architecture for real-time high-definition short-wave infrared streaming video processing and its field programmable gate array prototype

A wide-spectrum and high-speed CMOS system based on FPGA

Design of onboard image real-time processing system based on FPGA