Abstract:In recent years, Convolutional Neural Networks (CNNs) have been widely applied in computer vision and have achieved significant improvements in object detection tasks. Although there are many optimizing methods to speed up CNN-based detection algorithms, it is still difficult to deploy detection algorithms on real-time low-power systems. Field-Programmable Gate Array (FPGA) has been widely explored as a platform for accelerating CNN due to its promising performance, high energy efficiency, and flexibility. Previous works show that the energy consumption of CNN accelerators is dominated by the memory access. By fusing multiple layers in CNN, the intermediate data transfer can be reduced. However, previous accelerators with the cross-layer scheduling are designed for a particular CNN model. In addition to the memory access optimization, the Winograd algorithm can greatly improve the computational performance of convolution. In this article, to improve the flexibility of hardware, we design an instruction-driven CNN accelerator, supporting the Winograd algorithm and the cross-layer scheduling, for object detection. We modify the loop unrolling order of CNN, so that we can schedule a CNN across different layers with instructions and eliminate the intermediate data transfer. We propose a hardware architecture to support the instructions with Winograd computation units and reach the state-of-the-art energy efficiency. To deploy image detection algorithms onto the proposed accelerator with fixed-point computation units, we adopt the fixed-point fine-tune method, which can guarantee the accuracy of the detection algorithms. We evaluate our accelerator and scheduling policy on the Xilinx KU115 FPGA platform. The intermediate data transfer can be reduced by more than 90% on the VGG-D CNN model with the cross-layer strategy. Thus, the performance of our hardware accelerator reaches 1700GOP/s on the classification model VGG-D. We also implement a framework for object detection algorithms, which achieves 2.3× and 50× in energy efficiency compared with GPU and CPU, respectively. Compared with floating-point algorithms, the accuracy of the fixed-point detection algorithms only drops by less than 1%.

Detect and Replace: Efficient Soft Error Protection of FPGA-Based CNN Accelerators

Implementation of Highly Reliable Convolutional Neural Network with Low Overhead on Field-Programmable Gate Array

Soft Error Mitigation for Deep Convolution Neural Network on FPGA Accelerators.

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

Comparative study: AutoDPR-SEM for enhancing CNN reliability in SRAM-based FPGAs through autonomous reconfiguration

Systematic Reliability Evaluation of FPGA Implemented CNN Accelerators

Soft Error Tolerant Convolutional Neural Networks on FPGAs with Ensemble Learning

Hardware Trojan Attacks on the Reconfigurable Interconnections of Field-Programmable Gate Array-Based Convolutional Neural Network Accelerators and a Physically Unclonable Function-Based Countermeasure Detection Technique

Modeling the Effect of SEUs on the Configuration Memory of SRAM-FPGA Based CNN Accelerators

Instruction Driven Cross-layer CNN Accelerator for Fast Detection on FPGA

Towards Reliable AI Applications Via Algorithm-Based Fault Tolerance on NVDLA

Efficient Error-Tolerant Quantized Neural Network Accelerators

Mitigating Memory Wall Effects in CNN Engines with On-the-Fly Weights Generation

A flexible FPGA accelerator for convolutional neural networks

An Efficient CNN Accelerator Achieving High PE Utilization Using a Dense-/Sparse-Aware Redundancy Reduction Method and Data–Index Decoupling Workflow

Algorithm-Based Fault Tolerance for Convolutional Neural Networks

A Memory-Optimized and Energy-Efficient CNN Acceleration Architecture Based on FPGA.

High Power-Efficient and Performance-Density FPGA Accelerator for CNN-Based Object Detection

Communication-Aware and Resource-Efficient NoC-Based Architecture for CNN Acceleration

ITT-RNA: Imperfection Tolerable Training for RRAM-Crossbar-Based Deep Neural-Network Accelerator

A Novel Method for Soft Error Mitigation in FPGA using Adaptive Cross Parity Code