Abstract:Graph Neural Network (GNN) inference is used in many real-world applications. Data sparsity in GNN inference, including sparsity in the input graph and the GNN model, offer opportunities to further speed up inference. Also, many pruning techniques have been proposed for model compression that increase the data sparsity of GNNs. We propose Dynasparse, a comprehensive hardware-software codesign on FPGA to accelerate GNN inference through dynamic sparsity exploitation. For this, we decouple the GNN computation kernels from the basic computation primitives, and explore hardware-software codesign as follows: 1) Hardware design: We propose a novel unified accelerator design on FPGA to efficiently execute various computation primitives. We develop a customized soft processor that is tightly coupled with the accelerator to execute a runtime system. Moreover, we develop efficient hardware mechanisms to profile the data sparsity and perform on-the-fly data format transformation to prepare the input data for various computation primitives; 2) Software design: We develop a runtime system that works synergistically with the accelerator to perform dynamic kernel-to-primitive mapping based on data sparsity. We implement Dynasparse on a state-of-the-art FPGA platform, Xilinx Alveo U250, and evaluate the design using widely used GNN models (GCN, GraphSAGE, GIN and SGC). For the above GNN models and various input graphs, the proposed accelerator and dynamic kernel-to-primitive mapping reduces the inference latency by $3.73\times$ on the average compared with the static mapping strategies employed in the state-of-the-art GNN accelerators. Compared with state-of-the-art CPU (GPU) implementations, Dynasparse achieves up to $56.9\times$ ($2.37\times$) speedup in end-to-end latency.

SWIFT: Small-World-based Structural Pruning to Accelerate DNN Inference on FPGA

Small-world-based Structural Pruning for Efficient FPGA Inference of Deep Neural Networks

Regularization-Free Structural Pruning for GPU Inference Acceleration

An Efficient FPGA Accelerator Optimized for High Throughput Sparse CNN Inference.

Swift: A Multi-FPGA Framework for Scaling Up Accelerated Graph Analytics

An Efficient Hardware Accelerator for Structured Sparse Convolutional Neural Networks on FPGAs

FPGA Accelerator for CNN: an Exploration of the Kernel Structured Sparsity and Hybrid Arithmetic Computation

Optimizing the Deep Neural Networks by Layer-Wise Refined Pruning and the Acceleration on FPGA

SCRA: Systolic-Friendly DNN Compression and Reconfigurable Accelerator Co-Design

Work-in-Progress: A High-performance FPGA Accelerator for Sparse Neural Networks

Structured Term Pruning for Computational Efficient Neural Networks Inference

Dynasparse: Accelerating GNN Inference through Dynamic Sparsity Exploitation

Exploiting Weight-Level Sparsity in Channel Pruning with Low-Rank Approximation

Structured Weight Matrices-Based Hardware Accelerators in Deep Neural Networks: FPGAs and ASICs

SparseNN: A Performance-Efficient Accelerator for Large-Scale Sparse Neural Networks

An Efficient Hardware Accelerator for Sparse Convolutional Neural Networks on FPGAs

A Winograd-Based CNN Accelerator with a Fine-Grained Regular Sparsity Pattern

Highly Efficient Sparse Neural Network Computing - Hardware and Software Solutions.

SPAT: FPGA-based Sparsity-Optimized Spiking Neural Network Training Accelerator with Temporal Parallel Dataflow

A High-Performance Hardware Accelerator for Sparse Convolutional Neural Network on FPGA

A High-performance Inference Accelerator Exploiting Patterned Sparsity in CNNs