Abstract:Sparse matrix-vector multiplication (SpMV) plays a critical role in a wide range of linear algebra computations, particularly in scientific and engineering disciplines. However, the irregular memory access patterns, extensive memory usage, high bandwidth requirements, and underutilization of parallelism hinder the computational efficiency of SpMV on GPUs. In this paper, we propose a novel approach called block-wise dynamic mixed-precision (BDMP) to address these challenges. Our methodology involves partitioning the original matrix into uniformly sized blocks, with each block's size determined by considering architectural characteristics and accuracy requirements. Additionally, we dynamically assign precision to each block using a precision selection method that takes into account the value distribution of the original sparse matrix. We develop two distinct SpMV computation algorithms for BDMP: BDMP-PBP (Precision-based partitioning) and BDMP-TCKI (Tailored compression and kernel implementation). BDMP-PBP partitions the matrix into two independent matrices for separate computations based on block precision, offering flexibility for integration with other optimization techniques. Meanwhile, BDMP-TCKI focuses on achieving significant thread-level parallelism and memory utilization by tailoring an appropriate compressed storage format and kernel implementation for each block. We compare BDMP with NVIDIA's cuSPARSE library and three state-of-the-art SpMV methods, including SELLP, MergeBase, and BalanceCSR, using matrices from the University of Florida's SuiteSparse dataset collection. BDMP-PBP and BDMP-TCKI show average speedups up to 2.64 and 2.91 on Turing RTX 2080Ti, and up to 2.99 and 3.22 on Ampere A100. The results demonstrate that BDMP enables the optimization of computation speed without compromising the precision necessary for reliable results.

AMF-CSR: Adaptive Multi-Row Folding of CSR for SpMV on GPU.

LSRB-CSR: A Low Overhead Storage Format for SpMV on the GPU Systems

Efficient Algorithm Design of Optimizing SpMV on GPU.

An efficient sparse stiffness matrix vector multiplication using compressed sparse row storage format on AMD GPU

FastLoad: Speeding Up Data Loading of Both Sparse Matrix and Vector for SpMV on GPUs

Optimization of Sparse Matrix Computation for Algebraic Multigrid on GPUs

Optimizing sparse matrix-vector multiplication based on gpu

An Integral-equation-oriented Vectorized SpMV Algorithm and Its Application on CT Imaging Reconstruction

Implementation and optimization of SpMV algorithm based on SW26010P many-core processor and stored in BCSR format

TileSpMV: A Tiled Algorithm for Sparse Matrix-Vector Multiplication on GPUs

Diagonally-Addressed Matrix Nicknack: How to improve SpMV performance

Esspmv: an Embedded-FPGA-based Hardware Accelerator for Symmetric Sparse Matrix-Vector Multiplication.

An Optimized GP-GPU Warp Scheduling Algorithm for Sparse Matrix-Vector Multiplication

Block-wise dynamic mixed-precision for sparse matrix-vector multiplication on GPUs

TaiChi: A Hybrid Compression Format for Binary Sparse Matrix-Vector Multiplication on GPU

Towards Efficient SpMV on Sunway Manycore Architectures.

FPGA and GPU Implementation of Large Scale SpMV

AlphaSparse: Generating High Performance SpMV Codes Directly from Sparse Matrices

Sparse matrix partitioning for optimizing SpMV on CPU-GPU heterogeneous platforms

Scale-Free Sparse Matrix-Vector Multiplication on Many-Core Architectures

Atomic Reduction Based Sparse Matrix-Transpose Vector Multiplication on GPUs