Abstract:Sparse Tucker Decomposition (STD) algorithms learn a core tensor and a group of factor matrices to obtain an optimal low-rank representation feature for the \underline{H}igh-\underline{O}rder, \underline{H}igh-\underline{D}imension, and \underline{S}parse \underline{T}ensor (HOHDST). However, existing STD algorithms face the problem of intermediate variables explosion which results from the fact that the formation of those variables, i.e., matrices Khatri-Rao product, Kronecker product, and matrix-matrix multiplication, follows the whole elements in sparse tensor. The above problems prevent deep fusion of efficient computation and big data platforms. To overcome the bottleneck, a novel stochastic optimization strategy (SGD$\_$Tucker) is proposed for STD which can automatically divide the high-dimension intermediate variables into small batches of intermediate matrices. Specifically, SGD$\_$Tucker only follows the randomly selected small samples rather than the whole elements, while maintaining the overall accuracy and convergence rate. In practice, SGD$\_$Tucker features the two distinct advancements over the state of the art. First, SGD$\_$Tucker can prune the communication overhead for the core tensor in distributed settings. Second, the low data-dependence of SGD$\_$Tucker enables fine-grained parallelization, which makes SGD$\_$Tucker obtaining lower computational overheads with the same accuracy. Experimental results show that SGD$\_$Tucker runs at least 2$X$ faster than the state of the art.

Stochastic Iterative Hard Thresholding for Low-Tucker-Rank Tensor Recovery

Iterative Singular Tube Hard Thresholding Algorithms for Tensor Recovery

Tensor Denoising Using Low-Rank Tensor Train Decomposition

Stochastic Natural Thresholding Algorithms

Optimal low rank tensor recovery

An Iterative Reweighted Method for Tucker Decomposition of Incomplete Multiway Tensors

Projective Iterative Hard Thresholding Algorithm for Sparse Signal Recovery

An Iterative Reweighted Method for Tucker Decomposition of Incomplete Tensors

Minimum $n$-Rank Approximation via Iterative Hard Thresholding

Robust Low-rank Tensor Decomposition with the L2 Criterion

Low-Rank Tucker Approximation of a Tensor from Streaming Data

Randomized Iterative Hard Thresholding for Sparse Approximations

Guarantees of Fast Band Restricted Thresholding Algorithm for Low-Rank Matrix Recovery Problem

Stochastic IHT With Stochastic Polyak Step-Size for Sparse Signal Recovery

Low Multilinear Rank Approximation of Tensors and Application in Missing Traffic Data

Low-Rank and Sparse Enhanced Tucker Decomposition for Tensor Completion

Regularized Kaczmarz Algorithms for Tensor Recovery

Infrared Small Target Detection via Nonconvex Tensor Tucker Decomposition With Factor Prior

SGD_Tucker: A Novel Stochastic Optimization Strategy for Parallel Sparse Tucker Decomposition

Rank-Adaptive Tensor Completion Based on Tucker Decomposition