Abstract:This paper considers optimization problems on the Stiefel manifold XTX=Ip\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$X^{\mathsf{T}}X=I_p$$\end{document}, where X∈Rn×p\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$X\in \mathbb {R}^{n \times p}$$\end{document} is the variable and Ip\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$I_p$$\end{document} is the p\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$p$$\end{document}-by-p\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$p$$\end{document} identity matrix. A framework of constraint preserving update schemes is proposed by decomposing each feasible point into the range space of X\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$X$$\end{document} and the null space of XT\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$X^{\mathsf{T}}$$\end{document}. While this general framework can unify many existing schemes, a new update scheme with low complexity cost is also discovered. Then we study a feasible Barzilai–Borwein-like method under the new update scheme. The global convergence of the method is established with an adaptive nonmonotone line search. The numerical tests on the nearest low-rank correlation matrix problem, the Kohn–Sham total energy minimization and a specific problem from statistics demonstrate the efficiency of the new method. In particular, the new method performs remarkably well for the nearest low-rank correlation matrix problem in terms of speed and solution quality and is considerably competitive with the widely used SCF iteration for the Kohn–Sham total energy minimization.

Adaptive Localized Cayley Parametrization for Optimization Over Stiefel Manifold and Its Convergence Rate Analysis

Generalized Left-Localized Cayley Parametrization for Optimization with Orthogonality Constraints

Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

A Theory of the NEPv Approach for Optimization On the Stiefel Manifold

Convergence Analysis of Gradient Algorithms on Riemannian Manifolds Without Curvature Constraints and Application to Riemannian Mass

Global Optimization with Orthogonality Constraints Via Stochastic Diffusion on Manifold

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

Optimization without Retraction on the Random Generalized Stiefel Manifold

A Feasible Method for Optimization with Orthogonality Constraints

A Penalty-Free Infeasible Approach for a Class of Nonsmooth Optimization Problems Over the Stiefel Manifold

A Riemannian optimization method on the indefinite Stiefel manifold

Global Convergence Analysis of the Power Proximal Point and Augmented Lagrangian Method

Nonlinear Eigen-approach ADMM for Sparse Optimization on Stiefel Manifold

An Adaptive Regularized Proximal Newton-Type Methods for Composite Optimization over the Stiefel Manifold

Orthogonal Directions Constrained Gradient Method: from non-linear equality constraints to Stiefel manifold

A framework of constraint preserving update schemes for optimization on Stiefel manifold

Convergence and Optimality of the Adaptive Nonconforming Linear Element Method for the Stokes Problem

Parallelizable Algorithms for Optimization Problems with Orthogonality Constraints

Randomized Submanifold Subgradient Method for Optimization over Stiefel Manifolds