Abstract:In paper, we focus on the computation of the principal components for a general tensor, known as the tensor principal component analysis (PCA) problem. It has been proven that the general tensor PCA problem is reducible to its matricization form when the order is even. Usually it is considered as low-rank matrix completion problems theoretically. It is common to consider nuclear norm as a surrogate of the rank operator since it is the tightest convex lower bound of the rank operator under certain condition. However, most nuclear norm minimization based approaches involve numbers of singular value decomposition (SVD) operations. Given a matrix X ∈ R m × n , the time complexity of SVD operation is O(mn 2 ), which brings prohibitive computational burden to apply these methods in real applications. However, the problem is non-convex, and the proximal mapping associated with non-convex regularization is not easy to compute. It is always solved by the Linearized Alternating Direction Method of Multipliers (LADMM). Despite the success of LADMM in practice, it remains unknown if LADMM is convergent in solving such non-convex compositely regularized optimization. In this paper, we firstly present a detailed convergence analysis of the LADMM algorithm for solving non-convex compositely regularized optimization with a large class of non-convex penalties. Furthermore, we propose a new efficient and scalable algorithm for matrix principal component analysis called Proximal Linearized Alternating Direction Method of Multipliers for Principal Component Analysis(PLADMPCA). Different from traditional matrix factorization methods, PLADMPCA utilizes the linearization technique to formulate the matrix as an outer product of vectors, which greatly improves the computational efficacy compared to matrix factorization method. We empirically evaluate the proposed algorithm PLADMPCA on synthetic tensor data with different order. Results have shown that PLADMPCA have much better computational cost to matrix factorization based method. At the same time, it outperforms the state-of-art SVD based matrix completion algorithms by similar or better reconstruction accuracy with enormous advantages on efficiency.

A Geometric Algorithm for Contrastive Principal Component Analysis in High Dimension

Robust Principal Component Analysis Based on Maximum Correntropy Criterion

Deep Contrastive Principal Component Analysis Adaptive to Nonlinear Data

Identifying patterns differing between high-dimensional datasets with generalized contrastive PCA

A Covariance-Free Iterative Principal Component Analysis for High Dimensional and Large Scale Data

Dual Principal Component Pursuit: Probability Analysis and Efficient Algorithms

Diagonally-Dominant Principal Component Analysis

Dynamic Principal Component Analysis in High Dimensions

A Fast Data-Oriented Algorithm for Principal Component Analysis

Dynamic Principal Subspaces in High Dimensions

An Efficient Alternating Riemannian/Projected Gradient Descent Ascent Algorithm for Fair Principal Component Analysis

Sparse Discriminant PCA Based on Contrastive Learning and Class-Specificity Distribution.

Directional Principal Component Analysis for Image Matrix

Maximally Correlated Principal Component Analysis

The Algebra And Statistics Of Generalized Principal Component Analysis

Divergent Projection Analysis for Unsupervised Dimensionality Reduction

Robust Principal Component Analysis with Non-Greedy l-Norm Maximization.

Robust Principal Component Analysis: A Median of Means Approach

An Efficient Algorithm for Tensor Principal Component Analysis Via Proximal Linearized Alternating Direction Method of Multipliers

Hierarchical disjoint principal component analysis

Modified Principal Component Analysis: an Integration of Multiple Similarity Subspace Models.