Abstract:In recent years, graph neural networks (GNNs) have achieved impressive performance in various application fields by extracting information from graph-structured data. It contains extensive feature aggregation operations and has become a performance bottleneck, which can be abstracted as a specialized sparse-dense matrix multiplication (SpMM) operation. Previous works have leveraged the inner product or outer product to accelerate the feature aggregation process. However, inefficient execution leads to extremely unbalanced workloads and extensive intermediate data, hampering the performance of previous processors. So in this article, we demonstrate an algorithm/hardware co-optimization chance to enhance SpMM acceleration for GNNs. First, the algorithm part develops a dataflow-efficient SpMM algorithm that integrates three optimization methods to mitigate computation and memory access inefficiencies. Specifically, 1) the proposed equal-value partition method achieves fine-grained data partition and enables load balancing during data movement; 2) after observing the vertex aggregation phenomenon, a vertex-clustering optimization method is presented to enable significant data locality; and 3) the adaptive dataflow based on Gustavson’s algorithm is further implemented to enable the efficient distribution of sparse elements and improves computing resource utilization. Then, the hardware part features the proposed SpMM algorithm and customizes SDMA, a flexible and efficient accelerator to boost SpMM acceleration, which follows the adaptive dataflow to eliminate sparsity and explore the regular parallelism dimension. Finally, we prototype SDMA on the Xilinx Alveo U280 FPGA accelerator card. The results demonstrate that SDMA achieves $5.68\times $ – $14.68\times $ energy efficiency over the previous GPU implementations deployed on the Nvidia GTX 1080Ti and $1.32\times $ higher throughput over the state-of-the-art FPGA prototype.

Heuristic Adaptability to Input Dynamics for SpMM on GPUs

Adaptive SpMV/SpMSpV on GPUs for Input Vectors of Varied Sparsity

Efficient Sparse Matrix Kernels based on Adaptive Workload-Balancing and Parallel-Reduction

DTSpMV: An Adaptive SpMV Framework for Graph Analysis on GPUs

LO-SpMM: Low-cost Search for High-performance SpMM Kernels on GPUs

EC-SpMM: Efficient Compilation of SpMM Kernel on GPUs.

GE-SpMM: General-Purpose Sparse Matrix-Matrix Multiplication on GPUs for Graph Neural Networks

Algorithm/Hardware Co-Optimization for Sparsity-Aware SpMM Acceleration of GNNs

Accelerating Sparse Approximate Matrix Multiplication on GPUs

Fast Sparse Deep Neural Network Inference with Flexible SpMM Optimization Space Exploration

Auto-SpMV: Automated Optimizing SpMV Kernels on GPU

Accelerating approximate matrix multiplication for near-sparse matrices on GPUs

Design and Implementation of Adaptive SpMV Library for Multicore and Many-Core Architecture

Efficient Algorithm Design of Optimizing SpMV on GPU.

SpMMPlu: A Compiler Plug-in with Sparse IR for Efficient Sparse Matrix Multiplication.

Machine Learning-Based Kernel Selector for SpMV Optimization in Graph Analysis

IA-SpGEMM: an input-aware auto-tuning framework for parallel sparse matrix-matrix multiplication

Spada: Accelerating Sparse Matrix Multiplication with Adaptive Dataflow.

Yet Another Hybrid Strategy for Auto-tuning SpMV on GPUs

Optimization of Large-Scale Sparse Matrix-Vector Multiplication on Multi-GPU Systems

Predicting optimal sparse general matrix-matrix multiplication algorithm on GPUs