Abstract:Sparse matrix-vector multiplication (SpMV) plays a critical role in a wide range of linear algebra computations, particularly in scientific and engineering disciplines. However, the irregular memory access patterns, extensive memory usage, high bandwidth requirements, and underutilization of parallelism hinder the computational efficiency of SpMV on GPUs. In this paper, we propose a novel approach called block-wise dynamic mixed-precision (BDMP) to address these challenges. Our methodology involves partitioning the original matrix into uniformly sized blocks, with each block's size determined by considering architectural characteristics and accuracy requirements. Additionally, we dynamically assign precision to each block using a precision selection method that takes into account the value distribution of the original sparse matrix. We develop two distinct SpMV computation algorithms for BDMP: BDMP-PBP (Precision-based partitioning) and BDMP-TCKI (Tailored compression and kernel implementation). BDMP-PBP partitions the matrix into two independent matrices for separate computations based on block precision, offering flexibility for integration with other optimization techniques. Meanwhile, BDMP-TCKI focuses on achieving significant thread-level parallelism and memory utilization by tailoring an appropriate compressed storage format and kernel implementation for each block. We compare BDMP with NVIDIA's cuSPARSE library and three state-of-the-art SpMV methods, including SELLP, MergeBase, and BalanceCSR, using matrices from the University of Florida's SuiteSparse dataset collection. BDMP-PBP and BDMP-TCKI show average speedups up to 2.64 and 2.91 on Turing RTX 2080Ti, and up to 2.99 and 3.22 on Ampere A100. The results demonstrate that BDMP enables the optimization of computation speed without compromising the precision necessary for reliable results.

A method for accelerating low precision operations by sparse matrix multiplication

Predicting the Output Structure of Sparse Matrix Multiplication with Sampled Compression Ratio

Hardware-Efficient Mixed-Precision CP Tensor Decomposition

Acceleration of Approximate Matrix Multiplications on GPUs

Precision-Aware Iterative Algorithms Based on Group-Shared Exponents of Floating-Point Numbers

Block-wise dynamic mixed-precision for sparse matrix-vector multiplication on GPUs

Acceleration of complex matrix multiplication using arbitrary precision floating-point arithmetic

Dedicated Hardware Accelerators for Processing of Sparse Matrices and Vectors: A Survey

Accelerating Sparse DNN Models Without Hardware-Support Via Tile-Wise Sparsity

Accelerating Sparse Approximate Matrix Multiplication on GPUs

Mixed precision LU factorization on GPU tensor cores: reducing data movement and memory footprint

Performance evaluation of accelerated real and complex multiple-precision sparse matrix-vector multiplication

Accelerating approximate matrix multiplication for near-sparse matrices on GPUs

Performance Optimization for Sparse A(T)Ax in Parallel on Multicore Cpu

Optimization of Large-Scale Sparse Matrix-Vector Multiplication on Multi-GPU Systems

A Fine-Grained Sparse Accelerator for Multi-Precision DNN.

Floating-Point Multiply-Add with Approximate Normalization for Low-Cost Matrix Engines

Optimizing sparse general matrix–matrix multiplication for DCUs

Efficient Arbitrary Precision Acceleration for Large Language Models on GPU Tensor Cores

FlashSparse: Minimizing Computation Redundancy for Fast Sparse Matrix Multiplications on Tensor Cores

FLAASH: Flexible Accelerator Architecture for Sparse High-Order Tensor Contraction