Abstract:We consider convex relaxations for recovering low-rank tensors based on constrained minimization over a ball induced by the tensor nuclear norm, recently introduced in \cite{tensor_tSVD}. We build on a recent line of results that considered convex relaxations for the recovery of low-rank matrices and established that under a strict complementarity condition (SC), both the convergence rate and per-iteration runtime of standard gradient methods may improve dramatically. We develop the appropriate strict complementarity condition for the tensor nuclear norm ball and obtain the following main results under this condition: 1. When the objective to minimize is of the form $f(\mX)=g(\mA\mX)+\langle{\mC,\mX}\rangle$ , where $g$ is strongly convex and $\mA$ is a linear map (e.g., least squares), a quadratic growth bound holds, which implies linear convergence rates for standard projected gradient methods, despite the fact that $f$ need not be strongly convex. 2. For a smooth objective function, when initialized in certain proximity of an optimal solution which satisfies SC, standard projected gradient methods only require SVD computations (for projecting onto the tensor nuclear norm ball) of rank that matches the tubal rank of the optimal solution. In particular, when the tubal rank is constant, this implies nearly linear (in the size of the tensor) runtime per iteration, as opposed to super linear without further assumptions. 3. For a nonsmooth objective function which admits a popular smooth saddle-point formulation, we derive similar results to the latter for the well known extragradient method. An additional contribution which may be of independent interest, is the rigorous extension of many basic results regarding tensors of arbitrary order, which were previously obtained only for third-order tensors.

Supplementary Material of “ A Novel Sparsity Measure for Tensor Recovery ”

Tensor Recovery Based on a Novel Non-Convex Function Minimax Logarithmic Concave Penalty Function

A Novel Tensor Ring Sparsity Measurement for Image Completion

Tensor N-tubal rank and its convex relaxation for low-rank tensor recovery

A Scale-Invariant Relaxation in Low-Rank Tensor Recovery with an Application to Tensor Completion

Tensor recovery based on Bivariate Equivalent Minimax-Concave Penalty

Block-Sparse Tensor Recovery

Sparse Recovery: The Square of $\ell_1/\ell_2$ Norms

Tensor Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Tensors via Convex Optimization

Low-Rank Tensor Completion via Novel Sparsity-Inducing Regularizers

Convex recovery of tensors using nuclear norm penalization

Efficiency of First-Order Methods for Low-Rank Tensor Recovery with the Tensor Nuclear Norm Under Strict Complementarity

On The Synergy Between Nonconvex Extensions of The Tensor Nuclear Norm for Tensor Recovery

Tensor Graphical Model: Non-Convex Optimization and Statistical Inference

Robust tensor recovery with nonconvex and nonsmooth regularization

A Novel Tensor Factorization-Based Method with Robustness to Inaccurate Rank Estimation

Robust Low-Rank Tensor Recovery with Rectification and Alignment

Asymmetry Total Variation and Framelet Regularized Nonconvex Low-Rank Tensor Completion

Tensor completion via nonconvex tensor ring rank minimization with guaranteed convergence

Sparse Recovery: The Square of Norms

Robust Tensor CUR Decompositions: Rapid Low-Tucker-Rank Tensor Recovery with Sparse Corruptions