Abstract:Sparse principal component analysis (PCA), an important variant of PCA, attempts to find sparse loading vectors when conducting dimension reduction. This paper considers the nonsmooth Riemannian optimization problem associated with the ScoTLASS model1 for sparse PCA which can impose orthogonality and sparsity simultaneously. A Riemannian proximal method is proposed in the work of Chen et al.9 for the efficient solution of this optimization problem. In this paper, two acceleration schemes are introduced. First and foremost, we extend the FISTA method from the Euclidean space to the Riemannian manifold to solve sparse PCA, leading to the accelerated Riemannian proximal gradient method. Since the Riemannian optimization problem for sparse PCA is essentially nonconvex, a restarting technique is adopted to stabilize the accelerated method without sacrificing the fast convergence. Second, a diagonal preconditioner is proposed for the Riemannian proximal subproblem which can further accelerate the convergence of the Riemannian proximal methods. Numerical evaluations establish the computational advantages of the proposed methods over the existing proximal gradient methods on a manifold. Additionally, a short result concerning the convergence of the Riemannian subgradients of a sequence is established, which, together with the result in the work of Chen et al.,9 can show the stationary point convergence of the Riemannian proximal methods.

Sub‐sampled adaptive trust region method on Riemannian manifolds

Riemannian Trust Region Methods for Minimization

Convergence and worst-case complexity of adaptive Riemannian trust-region methods for optimization on manifolds

A Riemannian Subspace Limited-Memory SR1 Trust Region Method

Riemannian Trust Region Methods for SC$^1$ Minimization

A Riemannian Subspace BFGS Trust Region Method

A consistently adaptive trust-region method

Efficient Sampling on Riemannian Manifolds via Langevin MCMC

Inexact Adaptive Cubic Regularization Algorithms on Riemannian Manifolds and Application

A simple and practical adaptive trust-region method

Riemannian Adaptive Regularized Newton Methods with Hölder Continuous Hessians

Nonsmooth nonconvex optimization on Riemannian manifolds via bundle trust region algorithm

Fast convergence of trust-regions for non-isolated minima via analysis of CG on indefinite matrices

A Riemannian Alternating Descent Ascent Algorithmic Framework for Nonconvex-Linear Minimax Problems on Riemannian Manifolds

A TRUST-REGION METHOD FOR NONSMOOTH NONCONVEX OPTIMIZATION

An Improved Non-Monotonic Adaptive Trust Region Algorithm for Unconstrained Optimization

A general framework of Riemannian adaptive optimization methods with a convergence analysis

An Extension of Fast Iterative Shrinkage‐thresholding Algorithm to Riemannian Optimization for Sparse Principal Component Analysis

An Extension of Fast Iterative Shrinkage-thresholding to Riemannian Optimization for Sparse Principal Component Analysis

Decentralized Online Riemannian Optimization with Dynamic Environments

Function recovery on manifolds using scattered data