Abstract:This paper considers optimization problems on the Stiefel manifold XTX=Ip\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$X^{\mathsf{T}}X=I_p$$\end{document}, where X∈Rn×p\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$X\in \mathbb {R}^{n \times p}$$\end{document} is the variable and Ip\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$I_p$$\end{document} is the p\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$p$$\end{document}-by-p\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$p$$\end{document} identity matrix. A framework of constraint preserving update schemes is proposed by decomposing each feasible point into the range space of X\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$X$$\end{document} and the null space of XT\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$X^{\mathsf{T}}$$\end{document}. While this general framework can unify many existing schemes, a new update scheme with low complexity cost is also discovered. Then we study a feasible Barzilai–Borwein-like method under the new update scheme. The global convergence of the method is established with an adaptive nonmonotone line search. The numerical tests on the nearest low-rank correlation matrix problem, the Kohn–Sham total energy minimization and a specific problem from statistics demonstrate the efficiency of the new method. In particular, the new method performs remarkably well for the nearest low-rank correlation matrix problem in terms of speed and solution quality and is considerably competitive with the widely used SCF iteration for the Kohn–Sham total energy minimization.

Modified Partial-Update Newton-Type Algorithms for Unary Optimization

A New Algorithm For Unconstrained Optimization Problem With The Form Of Sum Of Squares Minimization

An improved inexact Newton's method for unary optimization

A New Modified Nonmonotone Adaptive Trust Region Method for Unconstrained Optimization

PCG-inexact Newton methods for unary optimization

A Modified Quasi-Newton Method for Structured Optimization with Partial Information on the Hessian

An Improved Non-Monotonic Adaptive Trust Region Algorithm for Unconstrained Optimization

Extended Newton Methods for Multiobjective Optimization: Majorizing Function Technique and Convergence Analysis

A framework of constraint preserving update schemes for optimization on Stiefel manifold

A Modified Trust Region Algorithm For Nonlinear Equations With New Updating Rule Of Trust Region Radius

A New Diagonal Quasi-Newton Updating Method With Scaled Forward Finite Differences Directional Derivative for Unconstrained Optimization

Stochastic Optimization for Nonconvex Problem with Inexact Hessian Matrix, Gradient, and Function

Stochastic Optimization for Non-convex Problem with Inexact Hessian Matrix, Gradient, and Function

Improved Algorithms Based on Trust Region Framework for Solving Unconstrained Derivative Free Optimization Problems

An Improved Inexact Newton Method

Newton Sketch: A Linear-time Optimization Algorithm with Linear-Quadratic Convergence

A trust region-type normal map-based semismooth Newton method for nonsmooth nonconvex composite optimization

An augmented Lagrangian trust-region method with inexact gradient evaluations to accelerate constrained optimization problems using model hyperreduction

On the Modified Trust Region Algorithm for Nonlinear Equations

Novel Alternating Update Method for Low Rank Approximation of Structured Matrices

Incremental Quasi-Newton Methods with Faster Superlinear Convergence Rates