Abstract:Tensor algebra finds applications in various domains, including machine learning applications, data analytics, and others. Spatial hardware accelerators are widely used to boost the performance of tensor algebra applications. It has a complex hardware architecture and rich design space. Prior approaches based on manual implementation lead to low programming productivity, making it hard to explore the large design space. In this article, we propose Tensorlib, a framework for generating spatial hardware accelerators for tensor algebra applications. Tensorlib is motivated by the observation that, tensor dataflows can be expressed with linear transformations, and they share common hardware modules which can be reused across different designs. Tensorlib first uses space-time transformation to explore different dataflows, which can compactly represent the hardware dataflow using a transformation matrix. Next, we identify the common structures of different dataflows and build parameterized hardware module templates. Our generation framework can select the needed hardware modules for each dataflow, connect the modules using a specified interconnection pattern, and automatically generate the complete hardware accelerator design. Tensorlib remarkably improves the productivity for the development and optimization of spatial hardware architecture, providing a rich design space with tradeoffs in performance, area, and power. Experiments show that Tensorlib can automatically generate hardware designs with different dataflows for a variety of tensor algebra programs. Tensorlib can achieve 318-MHz frequency and 786-GFLOP/s throughput for matrix multiplication kernel on Xilinx VU9P FPGA, which outperforms the state-of-the-art generators.

FreeTensor: A Free-Form DSL with Holistic Optimizations for Irregular Tensor Programs

FlexTensor

The tensor algebra compiler

LazyTensor: combining eager execution with domain-specific compilers

Enabling Tensor Language Model to Assist in Generating High-Performance Tensor Programs for Deep Learning.

TensorLib - A Spatial Accelerator Generation Framework for Tensor Algebra.

Optimizing Tensor Programs on Flexible Storage

Differentiating a Tensor Language

OLLIE: Derivation-based Tensor Program Optimizer

Automatic Generation of Spatial Accelerator for Tensor Algebra

High-Performance Generalized Tensor Operations

Domain-Specific Tensor Languages

FTuner: A Fast Dynamic Shape Tensors Program Auto-Tuner for Deep Learning Compilers

Compressing Structured Tensor Algebra

TSCompiler: Efficient Compilation Framework for Dynamic-Shape Models

ReACT: Redundancy-Aware Code Generation for Tensor Expressions.

High-Performance Tensor Learning Primitives Using GPU Tensor Cores

PowerFusion: A Tensor Compiler with Explicit Data Movement Description and Instruction-level Graph IR

Uncovering Nested Data Parallelism and Data Reuse in DNN Computation with FractalTensor

Automatic generation of efficient sparse tensor format conversion routines

EINNET: Optimizing Tensor Programs with Derivation-Based Transformations.