Abstract:This article presents GraphAGILE, a domain-specific FPGA-based overlay accelerator for graph neural network (GNN) inference. GraphAGILE consists of (1) a novel unified architecture design with an instruction set, and (2) a compiler built upon the instruction set that can quickly generate optimized code. Due to the proposed instruction set architecture (ISA) and the compiler, GraphAGILE does not require any FPGA reconfiguration when performing inference on various GNN models and input graphs. For the architecture design, we propose a novel hardware module named Adaptive Computation Kernel (ACK), that can execute various computation kernels of GNNs, including general matrix multiplication (GEMM), sparse-dense matrix multiplication (SpDMM), and sampled dense-dense matrix multiplication (SDDMM). The compiler takes the specifications of a GNN model and the graph meta data (e.g., the number of vertices and edges) as input, and generates a sequence of instructions for inference execution. We develop the following compiler optimizations to reduce inference latency: 1) computation order optimization that automatically reorders the computation graph to reduce the total computation complexity, 2) layer fusion that merges adjacent layers to reduce data communication volume, 3) data partitioning with a partition-centric execution scheme that partitions the input graph to fit the available on-chip memory of FPGA, 4) kernel mapping that automatically selects execution mode for ACK, and performs task scheduling to overlap computation with data communication and achieves dynamic load balance. We implement GraphAGILE on a state-of-the-art FPGA platform, Xilinx Alveo U250. GraphAGILE can execute widely used GNN models, including GCN, GAT, GIN, GraphSAGE, SGC and other GNN models supported by GraphGym. Experimental results show that GraphAGILE achieves up to $47.1\times$47.1× ($3.9\times$3.9×) reduction in end-to-end latency, including the latency of compilation and hardware execution, compared with the state-of-the-art implementations on CPU (GPU), and achieves up to $2.9\times$2.9× reduction in hardware execution latency compared with the state-of-the-art FPGA accelerators.

FTW-GAT: an FPGA-Based Accelerator for Graph Attention Networks with Ternary Weights

FPGAN: An FPGA Accelerator for Graph Attention Networks With Software and Hardware Co-Optimization

SH-GAT: Software-hardware co-design for accelerating graph attention networks on FPGA

NTGAT: A Graph Attention Network Accelerator with Runtime Node Tailoring

FP-GNN: Adaptive FPGA Accelerator for Graph Neural Networks

DyGA: A Hardware-Efficient Accelerator with Traffic-Aware Dynamic Scheduling for Graph Convolutional Networks.

LW-GCN: A Lightweight FPGA-based Graph Convolutional Network Accelerator

A Gather Accelerator for GNNs on FPGA Platform.

FAMOUS: Flexible Accelerator for the Attention Mechanism of Transformer on UltraScale+ FPGAs

AWB-GCN: A Graph Convolutional Network Accelerator with Runtime Workload Rebalancing

GATe: Streamlining Memory Access and Communication to Accelerate Graph Attention Network With Near-Memory Processing

Unified Accelerator for Attention and Convolution in Inference Based on FPGA

ACE-GCN: A Fast Data-driven FPGA Accelerator for GCN Embedding

FTA-GAN: A Computation-Efficient Accelerator for GANs with Fast Transformation Algorithm

Towards a Deep-Pipelined Architecture for Accelerating Deep GCN on a Multi-FPGA Platform

A Runtime-Adaptive Transformer Neural Network Accelerator on FPGAs

A Reconfigurable Accelerator for Generative Adversarial Network Training Based on FPGA

Exploring Efficient Acceleration Architecture for Winograd-Transformed Transposed Convolution of GANs on FPGAs

GraphAGILE: An FPGA-based Overlay Accelerator for Low-latency GNN Inference

Evaluating a New Attention Framework Based on Matrix Blocking for Attention Models on FPGAs.

Exploring Resource-Efficient Acceleration Algorithm for Transposed Convolution of GANs on FPGA