Abstract:Sparse Triangular Solve (SpTRSV) has long been an essential kernel in the field of scientific computing. Due to its low computational intensity and internal data dependencies, SpTRSV is hard to implement and optimize on GPUs. Based on our experimental observations, existing implementations on GPUs fail to achieve the optimal performance due to their sub-optimal parallelism setups and code implementations, and lack of consideration of the irregular data distribution. Moreover, their algorithm design lacks the adaptability to different input matrices, which may involve substantial manual efforts of algorithm redesigning and parameter tuning for performance consistency. In this work, we propose AG-SpTRSV, an automatic framework to optimize SpTRSV on GPUs, which provides high performance on various matrices while eliminating the costs of manual design. AG-SpTRSV abstracts the procedures of optimizing an SpTRSV kernel as a scheme and constructs a comprehensive optimization space based on it. By defining a unified code template and preparing code variants, AG-SpTRSV enables fine-grained dynamic parallelism and adaptive code optimizations to handle various tasks. Through computation graph transformation and multi-hierarchy heuristic scheduling, AG-SpTRSV generates schemes for task partitioning and mapping, which effectively address the issues of irregular data distribution and internal data dependencies. AG-SpTRSV searches for the best scheme to optimize the target kernel for the specific matrix. A learned lightweight performance model is also introduced to reduce search costs and provide an efficient end-to-end solution. Experimental results with SuiteSparse Matrix Collection on NVIDIA Tesla A100 and RTX 3080 Ti show that AG-SpTRSV outperforms state-of-the-art implementations with geometric average speedups of 2.12x ∼ 3.99x. With the performance model enabled, AG-SpTRSV can provide an efficient end-to-end solution, with preprocessing times ranging from 3.4 to 245 times of the execution time.

Soter: Analytical Tensor-Architecture Modeling and Automatic Tensor Program Tuning for Spatial Accelerators

Automatic Generation of Spatial Accelerator for Tensor Algebra

HAOTuner: A Hardware Adaptive Operator Auto-Tuner for Dynamic Shape Tensor Compilers

TensorLib - A Spatial Accelerator Generation Framework for Tensor Algebra.

Unifying Spatial Accelerator Compilation With Idiomatic and Modular Transformations

FTuner: A Fast Dynamic Shape Tensors Program Auto-Tuner for Deep Learning Compilers

T2S-Tensor: Productively Generating High-Performance Spatial Hardware for Dense Tensor Computations

Analyzing the Design Space of Spatial Tensor Accelerators on FPGAs

AMOS: enabling <u>a</u>utomatic <u>m</u>apping for tensor computations <u>o</u>n <u>s</u>patial accelerators with hardware abstraction

Sifter: an Efficient Operator Auto-Tuner with Speculative Design Space Exploration for Deep Learning Compiler

A High-Performance Sparse Tensor Algebra Compiler in Multi-Level IR

SparseAuto: An Auto-Scheduler for Sparse Tensor Computations Using Recursive Loop Nest Restructuring

Enabling Tensor Language Model to Assist in Generating High-Performance Tensor Programs for Deep Learning.

TensorIR: an Abstraction for Automatic Tensorized Program Optimization.

Optimized Spatial Architecture Mapping Flow for Transformer Accelerators

Sgap: Towards Efficient Sparse Tensor Algebra Compilation for GPU

TensorMap: A Deep RL-Based Tensor Mapping Framework for Spatial Accelerators

Programmatic Control of a Compiler for Generating High-performance Spatial Hardware

AG-SpTRSV: an Automatic Framework to Optimize Sparse Triangular Solve on GPUs

Efficient Auto-Tuning of Parallel Programs with Interdependent Tuning Parameters via Auto-Tuning Framework (ATF)

Tailors: Accelerating Sparse Tensor Algebra by Overbooking Buffer Capacity