Abstract:With the advent of high-dimensional data, tensor and tensor decomposition methods have received widespread attention in the field of data analysis. However, the high-dimensional and sparse characteristics of tensor data lead to the high computation complexity of algorithm, which became an obstacle to the actual application of the tensor decomposition algorithm. In this paper, we start from the high sparsity of real tensor, and give a GPU parallel sparse tensor CP decomposition algorithm called ParSCP-ALS. Introduction In the past ten years, the tensor and tensor decomposition have become popular in high-dimensional data analysis. The tensor decomposition method is similar to matrix decomposition, which can map high-dimensional data to a lower subspace, while reducing the dimension of data. So far, tensor and tensor decomposition have applied in many fields, such as signal processing, computer vision, data mining, graph analysis, etc.[1] Due to the high-dimensional nature of tensor, all tensor decomposition algorithms have high computation complexity. At the same time, the high-dimensional data, such as DBLP[2] and Moivlens dataset[3], are always sparse, and the size of data grows exponentially with the dimension. The efficiency of algorithm and sparseness of large-scale data prevent tensor decomposition for further application in practical. To solve these problems of tensor decomposition, many researchers have made several improvements. Papalexakis et al [4] proposed the ParCube algorithm and Antikainen et al [5] speed up the decomposition of non-negative tensors with GPU. The existing research results show that the GPU parallel computing method can effectively accelerate the tensor decomposition algorithm. In this paper, aiming at the high sparsity of tensor data, which is the most significant feature of real high-dimensional data, we proposed an algorithm called ParSCP-ALS, which improved the CP-ALS algorithm for CP decomposition and use GPU to accelerate the algorithm at the same time. The rest of this paper is organized as follow. Section 2 reviews the CP decomposition and CP-ALS algorithm. Section 3 introduces the proposed ParSCP-ALS algorithm, including the improvement of the traditional algorithm, the data structure for sparse data and the parallelization on GPU. Section 4 presents the experiment result on DBLP and Movielens dataset. Background Knowledge Tensor and matricization A tensor is a mathematical representation of multi-way arrays. The dimension of tensor is defined as order or mode. In this paper, we use X as a tensor, A as a matrix, and x as a vector. The

High-Performance Tensor Learning Primitives Using GPU Tensor Cores

High-Performance Tensor-Train Primitives Using GPU Tensor Cores

Scalable CP Decomposition for Tensor Learning using GPU Tensor Cores

cuFastTuckerPlus: A Stochastic Parallel Sparse FastTucker Decomposition Using GPU Tensor Cores

cuFasterTucker: A Stochastic Optimization Strategy for Parallel Sparse FastTucker Decomposition on GPU Platform

Woodpecker-DL: Accelerating Deep Neural Networks via Hardware-Aware Multifaceted Optimizations

Optimizing Sparse Tensor Times Matrix on GPUs

Hardware-Enabled Efficient Data Processing with Tensor-Train Decomposition

Optimizing Sparse Tensor Times Matrix on Multi-Core and Many-Core Architectures

A-Tucker: Fast Input-Adaptive and Matricization-Free Tucker Decomposition of Higher-Order Tensors on GPUs

TDC: Towards Extremely Efficient CNNs on GPUs via Hardware-Aware Tucker Decomposition

A Novel Parallel Algorithm for Sparse Tensor Matrix Chain Multiplication via TCU-Acceleration

A Parallel CP Decomposition Algorithm for Sparse Tensor

Hardware-Efficient Mixed-Precision CP Tensor Decomposition

cu_FastTucker: A Faster and Stabler Stochastic Optimization for Parallel Sparse Tucker Decomposition on Multi-GPUs

Tucker Tensor Decomposition on FPGA

Exploiting Hierarchical Parallelism and Reusability in Tensor Kernel Processing on Heterogeneous HPC Systems

Accelerating Large Language Model Training with Hybrid GPU-based Compression

TensorCache: Reconstructing Memory Architecture with SRAM-Based In-Cache Computing for Efficient Tensor Computations in GPGPUs

a-Tucker: Input-Adaptive and Matricization-Free Tucker Decomposition for Dense Tensors on CPUs and GPUs

GPNPU: Enabling Efficient Hardware-Based Direct Convolution with Multi-Precision Support in GPU Tensor Cores