Abstract:In this paper, to address the optimization problem on a compact matrix manifold, we introduce a novel algorithmic framework called the Transformed Gradient Projection (TGP) algorithm, using the projection onto this compact matrix manifold. Compared with the existing algorithms, the key innovation in our approach lies in the utilization of a new class of search directions and various stepsizes, including the Armijo, nonmonotone Armijo, and fixed stepsizes, to guide the selection of the next iterate. Our framework offers flexibility by encompassing the classical gradient projection algorithms as special cases, and intersecting the retraction-based line-search algorithms. Notably, our focus is on the Stiefel or Grassmann manifold, revealing that many existing algorithms in the literature can be seen as specific instances within our proposed framework, and this algorithmic framework also induces several new special cases. Then, we conduct a thorough exploration of the convergence properties of these algorithms, considering various search directions and stepsizes. To achieve this, we extensively analyze the geometric properties of the projection onto compact matrix manifolds, allowing us to extend classical inequalities related to retractions from the literature. Building upon these insights, we establish the weak convergence, convergence rate, and global convergence of TGP algorithms under three distinct stepsizes. In cases where the compact matrix manifold is the Stiefel or Grassmann manifold, our convergence results either encompass or surpass those found in the literature. Finally, through a series of numerical experiments, we observe that the TGP algorithms, owing to their increased flexibility in choosing search directions, outperform classical gradient projection and retraction-based line-search algorithms in several scenarios.

New vector transport operators extending a Riemannian CG algorithm to generalized Stiefel manifold with low-rank applications

New vector transport operators extending a Riemannian CG algorithm to generalized Stiefel manifold with low-rank applications

Vector Transport Free Riemannian LBFGS for Optimization on Symmetric Positive Definite Matrix Manifolds

Practical gradient and conjugate gradient methods on flag manifolds

Vector Transport-Free SVRG with General Retraction for Riemannian Optimization: Complexity Analysis and Practical Implementation

Convergence Analysis of Gradient Algorithms on Riemannian Manifolds Without Curvature Constraints and Application to Riemannian Mass

Convergence analysis of the transformed gradient projection algorithms on compact matrix manifolds

Two efficient nonlinear conjugate gradient methods for Riemannian manifolds

Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold

Coupling Matrix Manifolds and Their Applications in Optimal Transport

Optimization without Retraction on the Random Generalized Stiefel Manifold

A Riemannian optimization method on the indefinite Stiefel manifold

Riemannian Stochastic Proximal Gradient Methods for Nonsmooth Optimization over the Stiefel Manifold

Nonsmooth Optimization over the Stiefel Manifold and Beyond: Proximal Gradient Method and Recent Variants

Riemannian Optimization for Non-convex Euclidean Distance Geometry with Global Recovery Guarantees

Generalized Moving Least-Squares for Solving Vector-valued PDEs on Unknown Manifolds

Geometry in global coordinates in mechanics and optimal transport

Distributed Riemannian Stochastic Gradient Tracking Algorithm on the Stiefel Manifold

Global Convergence of Decentralized Retraction-Free Optimization on the Stiefel Manifold

Global Optimization with Orthogonality Constraints Via Stochastic Diffusion on Manifold