Abstract:In the emerging edge-computing scenarios, FPGAs have been widely adopted to accelerate convolutional neural network (CNN)–based image-processing applications, such as image classification, object detection, and image segmentation, and so on. A standard image-processing pipeline first decodes the collected compressed images from Internet of Things (IoTs) to RGB data, then feeds them into CNN engines to compute the results. Previous works mainly focus on optimizing the CNN inference parts. However, we notice that on the popular ZYNQ FPGA platforms, image decoding can also become the bottleneck due to the poor performance of embedded ARM CPUs. Even with a hardware accelerator, the decoding operations still incur considerable latency. Moreover, conventional RGB-based CNNs have too few input channels at the first layer, which can hardly utilize the high parallelism of CNN engines and greatly slows down the network inference. To overcome these problems, in this article, we propose FD-CNN, a novel CNN accelerator leveraging the partial-decoding technique to accelerate CNNs directly in the frequency domain. Specifically, we omit the most time-consuming IDCT (Inverse Discrete Cosine Transform) operations of image decoding and directly feed the DCT coefficients (i.e., the frequency data) into CNNs. By this means, the image decoder can be greatly simplified. Moreover, compared to the RGB data, frequency data has a narrower input resolution but has 64× more channels. Such an input shape is more hardware friendly than RGB data and can substantially reduce the CNN inference time. We then systematically discuss the algorithm, architecture, and command set design of FD-CNN. To deal with the irregularity of different CNN applications, we propose an image-decoding-aware design-space exploration (DSE) workflow to optimize the pipeline. We further propose an early stopping strategy to tackle the time-consuming progressive JPEG decoding. Comprehensive experiments demonstrate that FD-CNN achieves, on average, 3.24×, 4.29× throughput improvement, 2.55×, 2.54× energy reduction and 2.38×, 2.58× lower latency on ZC-706 and ZCU-102 platforms, respectively, compared to the baseline image-processing pipelines.

Adaptive FPGA NoC-based Architecture for Multispectral Image Correlation

Improved real-time correlation-based FPGA stereo vision system

FPGA Implementation of Non-Parametric Stereo Matching Algorithm

Cellular Neural Network (CENN) FPGA implementation using multi-level optimization

FPGA based cellular neural network optimization: from design space to system.

A Multi-Level-Optimization Framework for FPGA-Based Cellular Neural Network Implementation.

Evaluation and Design Space Exploration of a Time-Division Multiplexed NoC on FPGA for Image Analysis Applications

A Novel Parallel Architecture for Template Matching Based on Zero-Mean Normalized Cross-Correlation

An Efficient FPGA-Based Implementation for Quantized Remote Sensing Image Scene Classification Network

Colorimetric Sensor Arrays System Based on FPGA for Image Recognition

A Novel Reconfigurable Computing Architecture for Image Signal Processing Using Circuit-Switched NoC and Synchronous Dataflow Model.

SolarAccel: FPGA accelerated 2D cross-correlation of digital images: Application to solar adaptive optics

FPGA implementation of collaborative representation algorithm for real-time hyperspectral target detection

Research and Implementation of FPGA Image Compression Parallel Algorithm Based on CCSDS

Optimizing CNN-based Hyperspectral Image Classification on FPGAs

FPGA implementation of a 32x32 autocorrelator array for analysis of fast image series

Design of Fully Spectral CNNs for Efficient FPGA-Based Acceleration

Implementation of hyperspectral inversion algorithms on FPGA: Hardware comparison using High Level Synthesis

Accelerating Hyperspectral Anomaly Detection With Enhanced Multivariate Gaussianization Based on FPGA

FD-CNN: A Frequency-Domain FPGA Acceleration Scheme for CNN-Based Image-Processing Applications

FPGA Acceleration of Color Interpolation Algorithm Based on Gradient and Color Difference