Abstract:Canonical polyadic decomposition (CPD) is one of the most common tensor computations adopted in many scientific applications. The major bottleneck of CPD is matricized tensor times Khatri-Rao product (MTTKRP). To optimize the performance of MTTKRP, various sparse tensor formats have been proposed such as CSF and HiCOO. However, due to the spatial complexity of the tensors, no single format fits all tensors. To address this problem, we propose SpTFS, a framework that automatically predicts the optimal storage format for an input sparse tensor. Specifically, SpTFS leverages a set of sampling methods to lower the sparse tensor to fixed-sized matrices and sparsity features. In addition, SpTFS adopts both supervised learning based and unsupervised learning based methods to predict the optimal sparse tensor storage formats. For supervised learning, we propose TnsNet that combines convolution neural network (CNN) and the feature layer, which effectively captures the sparsity patterns of the input tensors. Once trained, the TnsNet can be used with either density or histogram representation of the input tensor for optimal format prediction. Whereas for unsupervised learning, we propose TnsClustering that consists of a feature encoder using convolutional layers and fully connected layers, and a K-means++ model to cluster sparse tensors for optimal tensor format prediction, without massively profiling on the hardware platform. SpTFS can use the above two models to predict the optimal tensor storage format for accelerating MTTKRP accurately. The experimental results show that both TnsNet and TnsClustering can achieve higher prediction accuracy and performance speedup compared to the state-of-the-art works.

TS-RTPM-Net: Data-driven Tensor Sketching for Efficient CP Decomposition

Faster Tensor Train Decomposition for Sparse Data

SpTFS: Sparse Tensor Format Selection for MTTKRP Via Deep Learning

RTP: Rethinking Tensor Parallelism with Memory Deduplication

Faster TKD: Towards Lightweight Decomposition for Large-Scale Tensors with Randomized Block Sampling

Fast Dtt - A Near Linear Algorithm For Decomposing A Tensor Into Factor Tensors

An Efficient Two-Sided Sketching Method for Large-Scale Tensor Decomposition Based on Transformed Domains

Practical sketching‐based randomized tensor ring decomposition

Improved Robust Tensor Principal Component Analysis Via Low-Rank Core Matrix

Input-aware Sparse Tensor Storage Format Selection for Optimizing MTTKRP

Tensor Grid Decomposition with Application to Tensor Completion.

Sparse MTTKRP Acceleration for Tensor Decomposition on GPU

Stacked Tucker Decomposition with Multi-Nonlinear Products for Remote Sensing Imagery Inpainting

Fast DTT

Robust Tensor CUR Decompositions: Rapid Low-Tucker-Rank Tensor Recovery with Sparse Corruption

TTPNet: A Neural Network for Travel Time Prediction Based on Tensor Decomposition and Graph Embedding

Tensor Decomposition Via Core Tensor Networks.

Latent Matrices for Tensor Network Decomposition and to Tensor Completion

Frequency-Weighted Robust Tensor Principal Component Analysis

CTD: Fast, accurate, and interpretable method for static and dynamic tensor decompositions

Shared Memory Parallelization of MTTKRP for Dense Tensors