Abstract:This article presents GraphAGILE, a domain-specific FPGA-based overlay accelerator for graph neural network (GNN) inference. GraphAGILE consists of (1) a novel unified architecture design with an instruction set, and (2) a compiler built upon the instruction set that can quickly generate optimized code. Due to the proposed instruction set architecture (ISA) and the compiler, GraphAGILE does not require any FPGA reconfiguration when performing inference on various GNN models and input graphs. For the architecture design, we propose a novel hardware module named Adaptive Computation Kernel (ACK), that can execute various computation kernels of GNNs, including general matrix multiplication (GEMM), sparse-dense matrix multiplication (SpDMM), and sampled dense-dense matrix multiplication (SDDMM). The compiler takes the specifications of a GNN model and the graph meta data (e.g., the number of vertices and edges) as input, and generates a sequence of instructions for inference execution. We develop the following compiler optimizations to reduce inference latency: 1) computation order optimization that automatically reorders the computation graph to reduce the total computation complexity, 2) layer fusion that merges adjacent layers to reduce data communication volume, 3) data partitioning with a partition-centric execution scheme that partitions the input graph to fit the available on-chip memory of FPGA, 4) kernel mapping that automatically selects execution mode for ACK, and performs task scheduling to overlap computation with data communication and achieves dynamic load balance. We implement GraphAGILE on a state-of-the-art FPGA platform, Xilinx Alveo U250. GraphAGILE can execute widely used GNN models, including GCN, GAT, GIN, GraphSAGE, SGC and other GNN models supported by GraphGym. Experimental results show that GraphAGILE achieves up to $47.1\times$47.1× ($3.9\times$3.9×) reduction in end-to-end latency, including the latency of compilation and hardware execution, compared with the state-of-the-art implementations on CPU (GPU), and achieves up to $2.9\times$2.9× reduction in hardware execution latency compared with the state-of-the-art FPGA accelerators.

Edge FPGA-based Onsite Neural Network Training.

A Near Memory Computing FPGA Architecture for Neural Network Acceleration

FPGA based cellular neural network optimization: from design space to system.

EF-Train: Enable Efficient On-device CNN Training on FPGA Through Data Reshaping for Online Adaptation or Personalization

SparkNoC: An energy-efficiency FPGA-based accelerator using optimized lightweight CNN for edge computing

DyGA: A Hardware-Efficient Accelerator with Traffic-Aware Dynamic Scheduling for Graph Convolutional Networks.

Optimizing Neural Network Inference in Edge Robotics by Harnessing FPGA Hardware Acceleration

An FPGA Implementation of GCN with Sparse Adjacency Matrix

Spike Trains Encoding Optimization for Spiking Neural Networks Implementation in FPGA

Acceleration of Deep Neural Network Training Using Field Programmable Gate Arrays

Design of Network-on-Chip-Based Restricted Coulomb Energy Neural Network Accelerator on FPGA Device

DGNN-Booster: A Generic FPGA Accelerator Framework For Dynamic Graph Neural Network Inference

Performance evaluation of acceleration of convolutional layers on OpenEdgeCGRA

Research on Convolutional Neural Network Inference Acceleration and Performance Optimization for Edge Intelligence

Efficient Edge AI: Deploying Convolutional Neural Networks on FPGA with the Gemmini Accelerator

Challenges in Energy-Efficient Deep Neural Network Training with FPGA.

Compressed CNN Training with FPGA-based Accelerator

Efficient Neural Networks on the Edge with FPGAs by Optimizing an Adaptive Activation Function

An Efficient Hardware Accelerator for Structured Sparse Convolutional Neural Networks on FPGAs

Accelerating Mobile Applications at the Network Edge with Software-Programmable FPGAs.

GraphAGILE: An FPGA-based Overlay Accelerator for Low-latency GNN Inference