Abstract:Field-programmable gate arrays (FPGAs) are an ideal candidate for accelerating graph neural networks (GNNs). However, the FPGA redeployment process is time-consuming when updating or switching between diverse GNN models across different applications. Existing GNN processors eliminate the need for FPGA redeployment when switching between different GNN models. However, adapting matrix multiplication types by switching processing units decreases hardware utilization. In addition, the bandwidth of DDR limits further improvements in hardware performance. This paper proposes a highly flexible FPGA-based overlay processor for GNN accelerations. Graph-OPU provides excellent flexibility and programmability for users, as the executable code of GNN models is automatically compiled and reloaded without requiring FPGA redeployment. First, we customize the compiler and instruction sets for the inference process of different GNN models. Second, we customize the datapath and optimize the data format in the microarchitecture to fully leverage the advantages of high bandwidth memory (HBM). Third, we design a unified matrix multiplication to handle both sparse-dense matrix multiplication (SpMM) and general matrix multiplication (GEMM), enhancing Graph-OPU performance. During Graph-OPU execution, the computational units are shared between SpMM and GEMM instead of being switched, which improves the hardware utilization. Finally, we implement a hardware prototype on the Xilinx Alveo U50 and test the mainstream GNN models using various datasets. Experimental results show that Graph-OPU achieves up to 1654 \(\times\) and 63 \(\times\) speedup, as well as up to 5305 \(\times\) and 422 \(\times\) energy efficiency boosts, compared to implementations on CPU and GPU, respectively. Graph-OPU outperforms state-of-the-art (SOTA) end-to-end overlay accelerators for GNN, reducing latency by an average of 1.36 \(\times\) and improving energy efficiency by 1.41 \(\times\) on average. Moreover, Graph-OPU exhibits an average 1.45 \(\times\) speed improvement in end-to-end latency over the SOTA GNN processor. Graph-OPU represents an in-depth study of an FPGA-based overlay processor for GNNs, offering high flexibility, speedup, and energy efficiency.

FNNG: A High-Performance FPGA-based Accelerator for K-Nearest Neighbor Graph Construction.

L-FNNG: Accelerating Large-Scale KNN Graph Construction on CPU-FPGA Heterogeneous Platform

A Near Memory Computing FPGA Architecture for Neural Network Acceleration

DGNN-Booster: A Generic FPGA Accelerator Framework For Dynamic Graph Neural Network Inference

GraphAGILE: An FPGA-based Overlay Accelerator for Low-latency GNN Inference

An optimized architecture for accelerating graph computing on FPGAs

Graph-OPU: A Highly Integrated FPGA-Based Overlay Processor for Graph Neural Networks

Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks

A Survey of FPGA-Based Neural Network Accelerator

Low-latency Mini-batch GNN Inference on CPU-FPGA Heterogeneous Platform

GNNHLS: Evaluating Graph Neural Network Inference via High-Level Synthesis

HitGNN: High-Throughput GNN Training Framework on CPU+Multi-FPGA Heterogeneous Platform

A survey of field programmable gate array (FPGA)-based graph convolutional neural network accelerators: challenges and opportunities

Communication-Aware and Resource-Efficient NoC-Based Architecture for CNN Acceleration

Multi-clusters: an Efficient Design Paradigm of NN Accelerator Architecture Based on FPGA

FFCNN: Fast FPGA based Acceleration for Convolution neural network inference

An FPGA-Based Accelerator for Graph Embedding using Sequential Training Algorithm

An FPGA-Based Accelerator for Neighborhood-Based Collaborative Filtering Recommendation Algorithms

OmniGraph: A Scalable Hardware Accelerator for Graph Processing

High-Performance FPGA-Based CNN Accelerator with Block-Floating-Point Arithmetic.