Abstract:With the advent of high-dimensional data, tensor and tensor decomposition methods have received widespread attention in the field of data analysis. However, the high-dimensional and sparse characteristics of tensor data lead to the high computation complexity of algorithm, which became an obstacle to the actual application of the tensor decomposition algorithm. In this paper, we start from the high sparsity of real tensor, and give a GPU parallel sparse tensor CP decomposition algorithm called ParSCP-ALS. Introduction In the past ten years, the tensor and tensor decomposition have become popular in high-dimensional data analysis. The tensor decomposition method is similar to matrix decomposition, which can map high-dimensional data to a lower subspace, while reducing the dimension of data. So far, tensor and tensor decomposition have applied in many fields, such as signal processing, computer vision, data mining, graph analysis, etc.[1] Due to the high-dimensional nature of tensor, all tensor decomposition algorithms have high computation complexity. At the same time, the high-dimensional data, such as DBLP[2] and Moivlens dataset[3], are always sparse, and the size of data grows exponentially with the dimension. The efficiency of algorithm and sparseness of large-scale data prevent tensor decomposition for further application in practical. To solve these problems of tensor decomposition, many researchers have made several improvements. Papalexakis et al [4] proposed the ParCube algorithm and Antikainen et al [5] speed up the decomposition of non-negative tensors with GPU. The existing research results show that the GPU parallel computing method can effectively accelerate the tensor decomposition algorithm. In this paper, aiming at the high sparsity of tensor data, which is the most significant feature of real high-dimensional data, we proposed an algorithm called ParSCP-ALS, which improved the CP-ALS algorithm for CP decomposition and use GPU to accelerate the algorithm at the same time. The rest of this paper is organized as follow. Section 2 reviews the CP decomposition and CP-ALS algorithm. Section 3 introduces the proposed ParSCP-ALS algorithm, including the improvement of the traditional algorithm, the data structure for sparse data and the parallelization on GPU. Section 4 presents the experiment result on DBLP and Movielens dataset. Background Knowledge Tensor and matricization A tensor is a mathematical representation of multi-way arrays. The dimension of tensor is defined as order or mode. In this paper, we use X as a tensor, A as a matrix, and x as a vector. The

Accelerated Doubly Stochastic Gradient Descent for Tensor CP Decomposition.

Accelerating Regularized Tensor Decomposition Using the Alternating Direction Method of Multipliers with Multiple Nesterov’s Extrapolations

Inertial Accelerated SGD Algorithms for Solving Large-Scale Lower-Rank Tensor CP Decomposition Problems

Accelerating regularized tensor decomposition using the alternating direction method of multipliers with multiple Nesterov's extrapolations

A momentum block-randomized stochastic algorithm for low-rank tensor cp decomposition

Efficient Nonnegative Tensor Decomposition Using Alternating Direction Proximal Method of Multipliers

Sparse Nonnegative CANDECOMP/PARAFAC Decomposition in Block Coordinate Descent Framework: A Comparison Study

A fast parallel tensor decomposition with optimal stochastic gradient descent: an application in structural damage identification

A block-randomized stochastic method with importance sampling for CP tensor decomposition

Inertial Accelerated Stochastic Mirror Descent for Large-Scale Generalized Tensor CP Decomposition

Stochastic Momentum Method with Double Acceleration for Regularized Empirical Risk Minimization

An inexactly accelerated algorithm for nonnegative tensor CP decomposition with the column unit constraints

Fast Learnings of Coupled Nonnegative Tensor Decomposition Using Optimal Gradient and Low-rank Approximation

Distributed-Memory Randomized Algorithms for Sparse Tensor CP Decomposition

Realistic Acceleration of Neural Networks with Fine-Grained Tensor Decomposition

A Parallel CP Decomposition Algorithm for Sparse Tensor

Escaping from Saddle Points --- Online Stochastic Gradient for Tensor Decomposition

Multi-stage stochastic gradient method with momentum acceleration

Randomized Online CP Decomposition

EA-ADMM: Noisy Tensor PARAFAC Decomposition Based on Element-Wise Average ADMM

SPALS: Fast Alternating Least Squares via Implicit Leverage Scores Sampling