Abstract:Learning tensor-train (TT) structure (a.k.a matrix product state (MPS) representation) from large-scale high-dimensional data has been a common task in big data analysis, deep learning, and quantum machine learning. However, tensor-train algorithms are compute-intensive, which hinders their real-world applications. In this paper, we present high-performance tensor-train primitives using GPU tensor cores and demonstrate three applications. First, we use GPU tensor cores to optimize tensor-train primitives, including tensor contraction, singular value decomposition, and data transfer and computing. Second, we utilize the optimized primitives to accelerate tensor-train decomposition algorithms for big data analysis. Further, we propose a shard mode for high-order tensor computations on multiple GPUs. Third, we apply the optimized primitives to accelerate the tensor-train layer for compressing deep neural networks. Last, we utilize the optimized primitives to accelerate a quantum machine learning algorithm called Density Matrix Renormalization Group (DMRG). In performance evaluations, our third-order TT tensor decomposition achieves up to 3.34× and 6.91× speedups over two popular libraries (namely T3F and tntorch) on an A100 GPU, respectively. The proposed sixth-order tensor-train decomposition achieves up to a speedup of 5.01× over T3F on multiple A100 GPUs. Our tensor-train layer for a fully connected neural network achieves a compression ratio of 65.3× at the cost of 0.3% drop in accuracy and a speedup of 1.53× over a PyTorch implementation on CUDA cores. The optimized DMRG algorithm achieves up to a speedup of 14.0× over TensorNetwork, indicating the potential of the optimized tensor primitives for the classical simulation of quantum machine learning algorithms.

NVIDIA A100 Tensor Core GPU: Performance and Innovation

Benchmarking the Nvidia GPU Lineage: From Early K80 to Modern A100 with Asynchronous Memory Transfers

Evaluating the Performance of NVIDIA's A100 Ampere GPU for Sparse Linear Algebra Computations

DaDianNao: A Machine-Learning Supercomputer

Heterogeneous Programming and Optimization of Gyrokinetic Toroidal Code and Large-Scale Performance Test on TH-1A.

Acceleration of Tensor-Product Operations with Tensor Cores

NVIDIA Tensor Core Programmability, Performance & Precision

High-resolution imaging on TPUs

HadaCore: Tensor Core Accelerated Hadamard Transform Kernel

TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings

High-Performance Tensor Learning Primitives Using GPU Tensor Cores

DSTC: Dual-Side Sparsity Tensor Core for DNNs Acceleration on Modern GPU Architectures

Modeling Deep Learning Accelerator Enabled GPUs

120 GOPS Photonic Tensor Core in Thin-film Lithium Niobate for Inference and in-situ Training

Accelerating Binarized Neural Networks via Bit-Tensor-Cores in Turing GPUs

Photonic tensor cores for machine learning

High-Performance Tensor-Train Primitives Using GPU Tensor Cores

G10: Enabling An Efficient Unified GPU Memory and Storage Architecture with Smart Tensor Migrations

TensorCIM: Digital Computing-in-Memory Tensor Processor with Multichip-Module-Based Architecture for Beyond-NN Acceleration

Graphcore C2 Card performance for image-based deep learning application: A Report

A Carbon-Nanotube-based Tensor Processing Unit