Abstract:Overlay is an effective approach for creating FPGA-based AI accelerators, enabling software-programmable specialized hardware datapaths to flexibly support various DNN operations. Traditional DNN overlays typically base their instruction set design on the von Neumann model but adapt them to be more coarse-grained. These instruction sets control execution at the layer granularity and impose restricted patterns for mapping computation and bandwidth resources. Such constraints cause inefficiencies from the imperfect match between supported execution patterns and diverse DNN layer shapes and types. This work proposes a Reconfigurable Stream Network architecture, a unique ISA abstraction tailored for flexible FPGA overlay execution at low cost, marking it as the first known FPGA design to support dynamic sequential linear layer pipelining. This novel architecture presents a datapath abstraction modeled after a specialized circuit-switched network with stateful functional units (FUs) as nodes and data streaming on edges. Programming a computation corresponds to triggering a network path in this stream-connected datapath. The program can individually control FUs to form paths that exploit both spatial and pipeline parallelism between independent and dependent concurrent computations. We present a proof-of-concept design RSN-XNN on the Versal VCK190. Evaluations show a 22x latency reduction for BERT compared to the state of the art, along with throughput improvements of 3.2x, 2.4x, 2.5x, and 2.8x for BERT, VIT, NCF, and MLP, respectively. RSN-XNN matches the latency of the T4 GPU with the same FP32 performance but only 18% of the memory bandwidth. Compared to the A100 GPU under the same 7nm process node, it achieves 2.1x/4.5x better operating/dynamic energy efficiency in FP32.

An FPGA-Based Neural Network Overlay for ADAS Supporting Multi-Model and Multi-Mode

A Near Memory Computing FPGA Architecture for Neural Network Acceleration

DLA: Compiler and FPGA Overlay for Neural Network Inference Acceleration

FPGA Oriented Lightweight Deep Learning Inference for Liver Cancer Segmentation

Multi-task ADAS system on FPGA

Real-Time Object Detection and Semantic Segmentation Hardware System with Deep Learning Networks

Accelerating autonomous vehicles: Harnessing FPGA power for deep learning advancements

Adaptive design and implementation of automatic modulation recognition accelerator

An Overview of FPGA Based Deep Learning Accelerators: Challenges and Opportunities.

Reconfigurable processor for deep learning in autonomous vehicles

An Edge Computing System with AMD Xilinx FPGA AI Customer Platform for Advanced Driver Assistance System

A Survey of FPGA Based Deep Learning Accelerators: Challenges and Opportunities

Addressing Architectural Obstacles for Overlay with Stream Network Abstraction

NAF: Deeper Network/Accelerator Co-Exploration for Customizing CNNs on FPGA

A General Neural Network Hardware Architecture on FPGA

A Survey of FPGA-Based Neural Network Accelerator

FusionAccel: A General Re-configurable Deep Learning Inference Accelerator on FPGA for Convolutional Neural Networks

A Convolutional Neural Network Accelerator Based on FPGA

FPGA Hardware Acceleration Design for Deep Learning

[DL] A Survey of FPGA-based Neural Network Inference Accelerators

The implementation of a Deep Recurrent Neural Network Language Model on a Xilinx FPGA