Abstract:Field-programmable gate arrays (FPGAs) are an ideal candidate for accelerating graph neural networks (GNNs). However, the FPGA redeployment process is time-consuming when updating or switching between diverse GNN models across different applications. Existing GNN processors eliminate the need for FPGA redeployment when switching between different GNN models. However, adapting matrix multiplication types by switching processing units decreases hardware utilization. In addition, the bandwidth of DDR limits further improvements in hardware performance. This paper proposes a highly flexible FPGA-based overlay processor for GNN accelerations. Graph-OPU provides excellent flexibility and programmability for users, as the executable code of GNN models is automatically compiled and reloaded without requiring FPGA redeployment. First, we customize the compiler and instruction sets for the inference process of different GNN models. Second, we customize the datapath and optimize the data format in the microarchitecture to fully leverage the advantages of high bandwidth memory (HBM). Third, we design a unified matrix multiplication to handle both sparse-dense matrix multiplication (SpMM) and general matrix multiplication (GEMM), enhancing Graph-OPU performance. During Graph-OPU execution, the computational units are shared between SpMM and GEMM instead of being switched, which improves the hardware utilization. Finally, we implement a hardware prototype on the Xilinx Alveo U50 and test the mainstream GNN models using various datasets. Experimental results show that Graph-OPU achieves up to 1654 \(\times\) and 63 \(\times\) speedup, as well as up to 5305 \(\times\) and 422 \(\times\) energy efficiency boosts, compared to implementations on CPU and GPU, respectively. Graph-OPU outperforms state-of-the-art (SOTA) end-to-end overlay accelerators for GNN, reducing latency by an average of 1.36 \(\times\) and improving energy efficiency by 1.41 \(\times\) on average. Moreover, Graph-OPU exhibits an average 1.45 \(\times\) speed improvement in end-to-end latency over the SOTA GNN processor. Graph-OPU represents an in-depth study of an FPGA-based overlay processor for GNNs, offering high flexibility, speedup, and energy efficiency.

Ugrapher: High-Performance Graph Operator Computation Via Unified Abstraction for Graph Neural Networks.

EnGN: A High-Throughput and Energy-Efficient Accelerator for Large Graph Neural Networks

A comprehensive survey on graph neural network accelerators

GraphTensor: Comprehensive GNN-Acceleration Framework for Efficient Parallel Processing of Massive Datasets

Graph-OPU: A Highly Integrated FPGA-Based Overlay Processor for Graph Neural Networks

VersaGNN: a Versatile accelerator for Graph neural networks

A Coordinated Strategy for GNN Combining Computational Graph and Operator Optimizations

Towards Efficient Large-Scale Graph Neural Network Computing.

GNNAdvisor: An Efficient Runtime System for GNN Acceleration on GPUs

FP-GNN: Adaptive FPGA Accelerator for Graph Neural Networks

AdaptGear: Accelerating GNN Training Via Adaptive Subgraph-Level Kernels on GPUs

Accelerating Generic Graph Neural Networks via Architecture, Compiler, Partition Method Co-Design

GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUs

ByteGNN: Efficient Graph Neural Network Training at Large Scale

Characterizing the Efficiency of Graph Neural Network Frameworks with a Magnifying Glass

Scalable and Efficient Full-Graph GNN Training for Large Graphs

CoGNN: An Algorithm-Hardware Co-Design Approach to Accelerate GNN Inference With Minibatch Sampling

Survey on Graph Neural Network Acceleration: an Algorithmic Perspective

ZIPPER: Exploiting Tile- and Operator-level Parallelism for General and Scalable Graph Neural Network Acceleration

Understanding GNN Computational Graph: A Coordinated Computation, IO, and Memory Perspective

Cached Operator Reordering: A Unified View for Fast GNN Training