Abstract:Deep learning is the amazing technology which has promoted the development of artificial intelligence and achieved many amazing successes in intelligent fields. Convolution-based layers (CLs), fully connected layers (FLs) and recurrent layers (RLs) are three types of layers in classic neural networks. Most intelligent tasks are implemented by the hybrid neural networks (hybrid-NNs), which are commonly composed of different layer-blocks (LBs) of CLs, FLs, and RLs. Because the CLs require the most computation in hybrid-NNs, many field-programmable gate array (FPGA)-based accelerators focus on CLs acceleration and have demonstrated great performance. However, the CLs accelerators lead to an underutilization of FPGA resources in the acceleration of the whole hybrid-NN. To fully exploit the logic resources and the memory bandwidth in the acceleration of CLs/FLs/RLs, we propose an FPGA resource efficient mapping mechanism for hybrid-NNs. The mechanism first improves the utilization of DSPs by integrating multiple small bit-width operations on one DSP. Then the LB-level spatial mapping is used to exploit the complementary features between different neural networks in the hybrid-NN. We evaluate the mapping mechanism by implementing four hybrid-NNs on Xilinx Virtex7 690T FPGA. The proposed mechanism achieves a peak performance of 1805.8 giga operations per second (GOPs). With the analysis on resource utilization and throughput, the proposed method exploits more computing power in FPGA and achieves up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4.13 \times$ </tex-math></inline-formula> higher throughput than the state-of-the-art acceleration.

MuDBN: An Energy-Efficient and High-Performance Multi-FPGA Accelerator for Deep Belief Networks.

A Near Memory Computing FPGA Architecture for Neural Network Acceleration

DaDianNao: A Machine-Learning Supercomputer

An All-Digital Compute-In-Memory FPGA Architecture for Deep Learning Acceleration

A fine-grained mixed precision DNN accelerator using a two-stage big-little core RISC-V MCU.

An Energy-Efficient Deep Belief Network Processor Based on Heterogeneous Multi-Core Architecture With Transposable Memory and On-Chip Learning

Multi-clusters: an Efficient Design Paradigm of NN Accelerator Architecture Based on FPGA

HybridDNN: A Framework for High-Performance Hybrid DNN Accelerator Design and Implementation.

Design Exploration of Multi-FPGAs for Accelerating Deep Learning

DGNN-Booster: A Generic FPGA Accelerator Framework For Dynamic Graph Neural Network Inference

Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework

Accelerating DNN Inference with Heterogeneous Multi-DPU Engines

High-Performance FPGA-based Accelerator for Bayesian Neural Networks

A high throughput acceleration for hybrid neural networks with efficient resource management on FPGA

Embedded Streaming Deep Neural Networks Accelerator With Applications

DeepBurning-MixQ: An Open Source Mixed-Precision Neural Network Accelerator Design Framework for FPGAs

A Power Efficient Neural Network Implementation on Heterogeneous FPGA and GPU Devices

Efficient Hardware Optimization Strategies For Deep Neural Networks Acceleration Chip

XFER: A Novel Design to Achieve Super-Linear Performance on Multiple FPGAs for Real-Time AI.

Accelerating Deep Neural Networks by Combining Block-Circulant Matrices and Low-Precision Weights

FPDeep: Scalable Acceleration of CNN Training on Deeply-Pipelined FPGA Clusters