Abstract:Sparse Triangular Solve (SpTRSV) has long been an essential kernel in the field of scientific computing. Due to its low computational intensity and internal data dependencies, SpTRSV is hard to implement and optimize on GPUs. Based on our experimental observations, existing implementations on GPUs fail to achieve the optimal performance due to their sub-optimal parallelism setups and code implementations, and lack of consideration of the irregular data distribution. Moreover, their algorithm design lacks the adaptability to different input matrices, which may involve substantial manual efforts of algorithm redesigning and parameter tuning for performance consistency. In this work, we propose AG-SpTRSV, an automatic framework to optimize SpTRSV on GPUs, which provides high performance on various matrices while eliminating the costs of manual design. AG-SpTRSV abstracts the procedures of optimizing an SpTRSV kernel as a scheme and constructs a comprehensive optimization space based on it. By defining a unified code template and preparing code variants, AG-SpTRSV enables fine-grained dynamic parallelism and adaptive code optimizations to handle various tasks. Through computation graph transformation and multi-hierarchy heuristic scheduling, AG-SpTRSV generates schemes for task partitioning and mapping, which effectively address the issues of irregular data distribution and internal data dependencies. AG-SpTRSV searches for the best scheme to optimize the target kernel for the specific matrix. A learned lightweight performance model is also introduced to reduce search costs and provide an efficient end-to-end solution. Experimental results with SuiteSparse Matrix Collection on NVIDIA Tesla A100 and RTX 3080 Ti show that AG-SpTRSV outperforms state-of-the-art implementations with geometric average speedups of 2.12x ∼ 3.99x. With the performance model enabled, AG-SpTRSV can provide an efficient end-to-end solution, with preprocessing times ranging from 3.4 to 245 times of the execution time.

IAP-SpTV: An Input-aware Adaptive Pipeline SpTV via GCN on CPU-GPU

A Pipeline Computing Method of SpTV for Three-Order Tensors on CPU and GPU

A Heterogeneous Parallel Computing Approach Optimizing SpTTM on CPU-GPU Via GCN

Aesptv: an Adaptive and Efficient Framework for Sparse Tensor-Vector Product Kernel on a High-Performance Computing Platform

GSpTC: High-Performance Sparse Tensor Contraction on CPU-GPU Heterogeneous Systems

Efficient Utilization of Multi-Threading Parallelism on Heterogeneous Systems for Sparse Tensor Contraction

AG-SpTRSV: an Automatic Framework to Optimize Sparse Triangular Solve on GPUs

A Pipelining-Based Heterogeneous Scheduling and Energy-Throughput Optimization Scheme for CNNs Leveraging Apache TVM

Enabling Data Movement and Computation Pipelining in Deep Learning Compiler

Efficient Processing of Sparse Tensor Decomposition via Unified Abstraction and PE-Interactive Architecture

Parallel Dynamic Sparse Approximate Inverse Preconditioning Algorithm on GPU

Optimizing Sparse Tensor Times Matrix on GPUs

Sgap: Towards Efficient Sparse Tensor Algebra Compilation for GPU

Optimizing Sparse Tensor Times Matrix on Multi-Core and Many-Core Architectures

Spwa: An Efficient Sparse Winograd Convolutional Neural Networks Accelerator On Fpgas

TaiChi: A Hybrid Compression Format for Binary Sparse Matrix-Vector Multiplication on GPU

TSTC: Enabling Efficient Training Via Structured Sparse Tensor Compilation

A Highly Configurable Hardware/Software Stack for DNN Inference Acceleration

Efficient Hardware Accelerator Based on Medium Granularity Dataflow for SpTRSV

TGPA: tile-grained pipeline architecture for low latency CNN inference

SPEED: Streaming Partition and Parallel Acceleration for Temporal Interaction Graph Embedding