Abstract:Sparse two-dimensional principal component analysis (2DPCA) is an effective dimensionality reduction and data processing technique, which has been successfully applied in face recognition. Sparse 2DPCA is based on 2D image matrices instead of 1D vectors so it can well exploit the structural information of image pixels. The sparse 2DPCA model is regarded as a non-convex and non-smooth composite optimization problem. The manifold proximal gradient algorithm (ManPG) that is efficient and robust can globally converge to an optimal point in solving this kind of problem on the Stiefel manifold, such as sparse PCA. ManPG uses the proximal gradient method for the non-smooth term of this problem, which yields a convex subproblem on the tangent space. By solving this subproblem, a descent direction on the tangent space is obtained. Then a new update point on the manifold is obtained via retraction. However, in solving large-scale problems, the computational complexity of ManPG is quite high since ManPG requires calculating the full gradient in each iteration. In order to reduce the computational complexity of each iteration, a more computationally efficient Riemannian proximal stochastic gradient descent algorithm (RPSGD) is proposed, which utilizes the stochastic gradient instead of the full gradient in ManPG. We prove that the iteration complexity of the proposed RPSGD is O(1/ϵ4) while it maintains the sublinear convergence for the large-scale sparse 2DPCA. The experimental results on face recognition and handwritten digits classification data sets show RPSGD has the advantages of small-time overhead, high test accuracy, good reconstructed image quality, and relatively sparse solution in solving the large-scale sparse 2DPCA.

Riemannian Proximal Stochastic Gradient Descent for Sparse 2DPCA.

An Extension of Fast Iterative Shrinkage‐thresholding Algorithm to Riemannian Optimization for Sparse Principal Component Analysis

An Extension of Fast Iterative Shrinkage-thresholding to Riemannian Optimization for Sparse Principal Component Analysis

An Efficient Alternating Riemannian/Projected Gradient Descent Ascent Algorithm for Fair Principal Component Analysis

Avoiding Optimal Mean Robust PCA/2DPCA with Non-Greedy L1-norm Maximization

Riemannian Stochastic Proximal Gradient Methods for Nonsmooth Optimization over the Stiefel Manifold

Manifold Quadratic Penalty Alternating Minimization for Sparse Principal Component Analysis

Riemannian Proximal Gradient Methods (extended Version)

On the Suboptimality of Proximal Gradient Descent for $\ell^{0}$ Sparse Approximation

An Alternating Manifold Proximal Gradient Method for Sparse Principal Component Analysis and Sparse Canonical Correlation Analysis

Self-Paced Two-dimensional PCA.

Sparse principal component analysis via regularized low rank matrix approximation

Riemannian proximal gradient methods

Robust principal component analysis for sparse face recognition

Robust 2D Principal Component Analysis: A Structured Sparsity Regularized Approach.

Nonconvex Regularized Robust PCA Using the Proximal Block Coordinate Descent Algorithm

A Riemannian Alternating Descent Ascent Algorithmic Framework for Nonconvex-Linear Minimax Problems on Riemannian Manifolds

Robust High-Order Manifold Constrained Sparse Principal Component Analysis for Image Representation

Advanced Variations of Two-Dimensional Principal Component Analysis for Face Recognition

Cosine 2DPCA With Weighted Projection Maximization

2D-PCA Representation and Sparse Representation for Image Recognition