Abstract:Field-programmable gate arrays (FPGAs) are an ideal candidate for accelerating graph neural networks (GNNs). However, the FPGA redeployment process is time-consuming when updating or switching between diverse GNN models across different applications. Existing GNN processors eliminate the need for FPGA redeployment when switching between different GNN models. However, adapting matrix multiplication types by switching processing units decreases hardware utilization. In addition, the bandwidth of DDR limits further improvements in hardware performance. This paper proposes a highly flexible FPGA-based overlay processor for GNN accelerations. Graph-OPU provides excellent flexibility and programmability for users, as the executable code of GNN models is automatically compiled and reloaded without requiring FPGA redeployment. First, we customize the compiler and instruction sets for the inference process of different GNN models. Second, we customize the datapath and optimize the data format in the microarchitecture to fully leverage the advantages of high bandwidth memory (HBM). Third, we design a unified matrix multiplication to handle both sparse-dense matrix multiplication (SpMM) and general matrix multiplication (GEMM), enhancing Graph-OPU performance. During Graph-OPU execution, the computational units are shared between SpMM and GEMM instead of being switched, which improves the hardware utilization. Finally, we implement a hardware prototype on the Xilinx Alveo U50 and test the mainstream GNN models using various datasets. Experimental results show that Graph-OPU achieves up to 1654 \(\times\) and 63 \(\times\) speedup, as well as up to 5305 \(\times\) and 422 \(\times\) energy efficiency boosts, compared to implementations on CPU and GPU, respectively. Graph-OPU outperforms state-of-the-art (SOTA) end-to-end overlay accelerators for GNN, reducing latency by an average of 1.36 \(\times\) and improving energy efficiency by 1.41 \(\times\) on average. Moreover, Graph-OPU exhibits an average 1.45 \(\times\) speed improvement in end-to-end latency over the SOTA GNN processor. Graph-OPU represents an in-depth study of an FPGA-based overlay processor for GNNs, offering high flexibility, speedup, and energy efficiency.

LTrans-OPU: A Low-Latency FPGA-Based Overlay Processor for Transformer Networks

FET-OPU: A Flexible and Efficient FPGA-Based Overlay Processor for Transformer Networks

TransFRU: Efficient Deployment of Transformers on FPGA with Full Resource Utilization

FTRANS: Energy-Efficient Acceleration of Transformers using FPGA

Fitop-Trans: Maximizing Transformer Pipeline Efficiency Through Fixed-Length Token Pruning on FPGA

A Runtime-Adaptive Transformer Neural Network Accelerator on FPGAs

Graph-OPU: A Highly Integrated FPGA-Based Overlay Processor for Graph Neural Networks

A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining

Accelerating Framework of Transformer by Hardware Design and Model Compression Co-Optimization

Uni-OPU: an FPGA-Based Uniform Accelerator for Convolutional and Transposed Convolutional Networks

ProTEA: Programmable Transformer Encoder Acceleration on FPGA

Enhancing Long Sequence Input Processing in FPGA-Based Transformer Accelerators Through Attention Fusion

TATAA: Programmable Mixed-Precision Transformer Acceleration with a Transformable Arithmetic Architecture

FAMOUS: Flexible Accelerator for the Attention Mechanism of Transformer on UltraScale+ FPGAs

PP-Transformer: Enable Efficient Deployment of Transformers Through Pattern Pruning

A 22nm 54.94TFLOPS/W Transformer Fine-Tuning Processor with Exponent-Stationary Re-Computing, Aggressive Linear Fitting, and Logarithmic Domain Multiplicating

HLSTransform: Energy-Efficient Llama 2 Inference on FPGAs Via High Level Synthesis

Hardware-friendly compression and hardware acceleration for transformer: A survey

DTrans: A Dataflow-transformation FPGA Accelerator with Nonlinear-operators Fusion Aiming for the Generative Model

Improving Transformer Inference Through Optimized Non-Linear Operations with Quantization-Approximation-Based Strategy

An FPGA-Based Reconfigurable Accelerator for Convolution-Transformer Hybrid EfficientViT