Abstract:Field-programmable gate arrays (FPGAs) are an ideal candidate for accelerating graph neural networks (GNNs). However, the FPGA redeployment process is time-consuming when updating or switching between diverse GNN models across different applications. Existing GNN processors eliminate the need for FPGA redeployment when switching between different GNN models. However, adapting matrix multiplication types by switching processing units decreases hardware utilization. In addition, the bandwidth of DDR limits further improvements in hardware performance. This paper proposes a highly flexible FPGA-based overlay processor for GNN accelerations. Graph-OPU provides excellent flexibility and programmability for users, as the executable code of GNN models is automatically compiled and reloaded without requiring FPGA redeployment. First, we customize the compiler and instruction sets for the inference process of different GNN models. Second, we customize the datapath and optimize the data format in the microarchitecture to fully leverage the advantages of high bandwidth memory (HBM). Third, we design a unified matrix multiplication to handle both sparse-dense matrix multiplication (SpMM) and general matrix multiplication (GEMM), enhancing Graph-OPU performance. During Graph-OPU execution, the computational units are shared between SpMM and GEMM instead of being switched, which improves the hardware utilization. Finally, we implement a hardware prototype on the Xilinx Alveo U50 and test the mainstream GNN models using various datasets. Experimental results show that Graph-OPU achieves up to 1654 \(\times\) and 63 \(\times\) speedup, as well as up to 5305 \(\times\) and 422 \(\times\) energy efficiency boosts, compared to implementations on CPU and GPU, respectively. Graph-OPU outperforms state-of-the-art (SOTA) end-to-end overlay accelerators for GNN, reducing latency by an average of 1.36 \(\times\) and improving energy efficiency by 1.41 \(\times\) on average. Moreover, Graph-OPU exhibits an average 1.45 \(\times\) speed improvement in end-to-end latency over the SOTA GNN processor. Graph-OPU represents an in-depth study of an FPGA-based overlay processor for GNNs, offering high flexibility, speedup, and energy efficiency.

FastONN -- Python based open-source GPU implementation for Operational Neural Networks

InstantTrace: Fast Parallel Neuron Tracing on GPUs

Self-Organized Operational Neural Networks with Generative Neurons

Fast Convolutional Nets With fbfft: A GPU Performance Evaluation

Operational neural networks

On-chip 4F-System Based on Concave Mirrors for Optical Neural Networks

FastGL: A GPU-Efficient Framework for Accelerating Sampling-Based GNN Training at Large Scale

FastSNN: A CUDA-Based Programming Framework for Rapid Training SNNs

GLP4NN: A Convergence-invariant and Network-agnostic Light-weight Parallelization Framework for Deep Neural Networks on Modern GPUs

A parallel Fortran framework for neural networks and deep learning

Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANN

Graph Neural Network-Accelerated Network-Reconfigured Optimal Power Flow

Training Large-Scale Optical Neural Networks with Two-Pass Forward Propagation

Free-Space Optical Spiking Neural Network

Utilizing cloud FPGAs towards the open neural network standard

Graph-OPU: A Highly Integrated FPGA-Based Overlay Processor for Graph Neural Networks

Implementation of Optical Deep Neural Networks using the Fabry-Perot Interferometer

Fastensor: Optimise the Tensor I/O Path from SSD to GPU for Deep Learning Training

MaxK-GNN: Towards Theoretical Speed Limits for Accelerating Graph Neural Networks Training

All-Optical Recurrent Neural Network With Reconfigurable Activation Function

Physics-Informed Deep Neural Operator Networks