Abstract:In the digital era, the prevalence of low-quality images contrasts with the widespread use of high-definition displays, primarily due to low-resolution cameras and compression technologies. Image super-resolution (SR) techniques, particularly those leveraging deep learning, aim to enhance these images for high-definition presentation. However, real-time execution of deep neural network (DNN)-based SR methods at the edge poses challenges due to their high computational and storage requirements. To address this, field-programmable gate arrays (FPGAs) have emerged as a promising platform, offering flexibility, programmability, and adaptability to evolving models. Previous FPGA-based SR solutions have focused on reducing computational and memory costs through aggressive simplification techniques, often sacrificing the quality of the reconstructed images. This paper introduces a novel SR network specifically designed for edge applications, which maintains reconstruction performance while managing computation costs effectively. Additionally, we propose an architectural design that enables the real-time and end-to-end inference of the proposed SR network on embedded FPGAs. Our key contributions include a tailored SR algorithm optimized for embedded FPGAs, a DSP-enhanced design that achieves a significant four-fold speedup, a novel scalable cache strategy for handling large feature maps, optimization of DSP cascade consumption, and a constraint optimization approach for resource allocation. Experimental results demonstrate that our FPGA-specific accelerator surpasses existing solutions, delivering superior throughput, energy efficiency, and image quality.

High-performance architecture for real-time high-definition short-wave infrared streaming video processing and its field programmable gate array prototype

High speed video signal acquisition and processing system based on FPGA technique

FPGA-based Design of Real-time Video Processing Platform

FPGA implementation of full HD real-time depth estimation

FPGA Implementation of Feature Detection Algorithm Based on High Level Synthesis

A low delay highly dynamic range infrared imaging system for complex scenes based on FPGA

Video Stream Processing on a High Performance Reconfigurable Architecture

Acceleration method of infrared image detail enhancement on FPGA

Real-time video processing model for stereoscopic display system based on FPGA

An Embedded System for Short Wave Infrared Image Enhancement and 3G-SDI Display

Design and Implementation of Infrared Image Preprocessing System Based on FPGA

Design of A High-speed Infrared Image Processing Platform Based on the Multi-core DSP and FPGA

A Kind of Real-Time Infrared Image Enhancement Algorithm

FPGA-Based High-Speed Infrared Image Acquisition and Transmission Design

IRFPA image detail enhancement and dynamic compression technique based on FPGA

Multi-Scale FPGA-Based Infrared Image Enhancement by Using RGF and CLAHE

A High-Performance Accelerator for Real-Time Super-Resolution on Edge FPGAs

Lightweight and Real-Time Infrared Image Processor Based on FPGA

High-Speed Video Transfer and Real-Time Infrared Spots Detection Based on FPGA

Design of Real-time Infrared Image Processing System Based on FPGA and DSP Framework

VLSI Architecture Exploration of Guided Image Filtering for 1080p@60hz Video Processing