Abstract:Graph Neural Network (GNN) inference is used in many real-world applications. Data sparsity in GNN inference, including sparsity in the input graph and the GNN model, offer opportunities to further speed up inference. Also, many pruning techniques have been proposed for model compression that increase the data sparsity of GNNs. We propose Dynasparse, a comprehensive hardware-software codesign on FPGA to accelerate GNN inference through dynamic sparsity exploitation. For this, we decouple the GNN computation kernels from the basic computation primitives, and explore hardware-software codesign as follows: 1) Hardware design: We propose a novel unified accelerator design on FPGA to efficiently execute various computation primitives. We develop a customized soft processor that is tightly coupled with the accelerator to execute a runtime system. Moreover, we develop efficient hardware mechanisms to profile the data sparsity and perform on-the-fly data format transformation to prepare the input data for various computation primitives; 2) Software design: We develop a runtime system that works synergistically with the accelerator to perform dynamic kernel-to-primitive mapping based on data sparsity. We implement Dynasparse on a state-of-the-art FPGA platform, Xilinx Alveo U250, and evaluate the design using widely used GNN models (GCN, GraphSAGE, GIN and SGC). For the above GNN models and various input graphs, the proposed accelerator and dynamic kernel-to-primitive mapping reduces the inference latency by $3.73\times$ on the average compared with the static mapping strategies employed in the state-of-the-art GNN accelerators. Compared with state-of-the-art CPU (GPU) implementations, Dynasparse achieves up to $56.9\times$ ($2.37\times$) speedup in end-to-end latency.

Factor Graph Inference Engine on the SpiNNaker Neural Computing System

Factor Graph Neural Networks

Factor Machine: Mixed-signal Architecture for Fine-Grained Graph-Based Computing

Neural Enhanced Belief Propagation on Factor Graphs

Structural Optimization of Factor Graphs for Symbol Detection via Continuous Clustering and Machine Learning

Learned Factor Graphs for Inference from Stationary Time Sequences

Spike Trains Encoding Optimization for Spiking Neural Networks Implementation in FPGA

GNNHLS: Evaluating Graph Neural Network Inference via High-Level Synthesis

Accelerating Large Sparse Neural Network Inference using GPU Task Graph Parallelism

Factor Graph-Based Planning as Inference for Autonomous Vehicle Racing

Factorizable Net: An Efficient Subgraph-based Framework for Scene Graph Generation

Local Message Passing on Frustrated Systems

Integer Factorization with Stochastic Spiking in Percolating Networks of Nanoparticles

FactorSim: Generative Simulation via Factorized Representation

Data-Driven Factor Graphs for Deep Symbol Detection

Dynasparse: Accelerating GNN Inference through Dynamic Sparsity Exploitation

Scheduling Computation Graphs of Deep Learning Models on Manycore CPUs

GNNIE: GNN Inference Engine with Load-balancing and Graph-Specific Caching

FAST: A simulation framework for solving large-scale probabilistic inverse problems in nano-biomolecular circuits

Streaming Probabilistic Deep Tensor Factorization

Spiker+: a framework for the generation of efficient Spiking Neural Networks FPGA accelerators for inference at the edge