Abstract:Sparse Matrix-Matrix Multiplication (SpMM) has served as fundamental components in various domains. Many previous studies exploit GPUs for SpMM acceleration because GPUs provide high bandwidth and parallelism. We point out that a static design does not always improve the performance of SpMM on different input data (e.g., >85% performance loss with a single algorithm). In this paper, we consider the challenge of input dynamics from a novel auto-tuning perspective, while following issues remain to be solved: (1) Orthogonal design principles considering sparsity. Orthogonal design principles for such a sparse problem should be extracted to form different algorithms, and further used for performance tuning. (2) Nontrivial implementations in the algorithm space. Combining orthogonal design principles to create new algorithms needs to tackle with new challenges like thread race handling. (3) Heuristic adaptability to input dynamics. The heuristic adaptability is required to dynamically optimize code for input dynamics. To tackle these challenges, we first propose a novel three-loop model to extract orthogonal design principles for SpMM on GPUs. The model not only covers previous SpMM designs, but also comes up with new designs absent from previous studies. We propose techniques like conditional reduction to implement algorithms missing in previous studies. We further propose DA-SpMM, a Data-Aware heuristic GPU kernel for SpMM. DA-SpMM adaptively optimizes code considering input dynamics. Extensive experimental results show that, DA-SpMM achieves 1.26x similar to 1.37x speedup compared with the best NVIDIA cuSPARSE algorithm on average, and brings up to 5.59x end-to-end speedup to Graph Neural Networks.

Performance Evaluations of Multiple GPUs based on MPI Environments

Performance Portability of Sparse Block Diagonal Matrix Multiple Vector Multiplications on GPUs

Efficient sparse-matrix multi-vector product on GPUs

Improvement of Sparse Matrix-Vector Multiplication on GPU

A New Sparse Matrix Vector Multiplication GPU Algorithm Designed for Finite Element Problems

Optimizing sparse matrix-vector multiplication based on gpu

Sparse matrix partitioning for optimizing SpMV on CPU-GPU heterogeneous platforms

Parallel optimization for sparse matrix-vector on GPU

Exploring the Diversity of Multiple Job Deployments over GPUs for Efficient Resource Sharing

Feature-based SpMV Performance Analysis on Contemporary Devices

Batched matrix operations on distributed GPUs with application in theoretical physics

Efficient Algorithm Design of Optimizing SpMV on GPU.

Accelerating Sparse Approximate Matrix Multiplication on GPUs

High Performance Computing Via a GPU

An Evaluation and Comparison of GPU Hardware and Solver Libraries for Accelerating the OPM Flow Reservoir Simulator

Multi-GPU thermal lattice Boltzmann simulations using OpenACC and MPI

Performance evaluation and analysis of sparse matrix and graph kernels on heterogeneous processors

Heuristic Adaptability to Input Dynamics for SpMM on GPUs

RDMA-Based Algorithms for Sparse Matrix Multiplication on GPUs

Parallel Efficient Sparse Matrix-Matrix Multiplication on Multicore Platforms

ParvaGPU: Efficient Spatial GPU Sharing for Large-Scale DNN Inference in Cloud Environments