Abstract:<span>Updating the augmented Lagrangian multiplier by closed-form expression yields efficient first-order infeasible approach for optimization problems with orthogonality constraints. Hence, parallelization becomes tractable in solving this type of problems. Inspired by this closed-form updating scheme, we propose a novel penalty function with compact convex constraints (PenC). We show that PenC can act as an exact penalty model which shares the same global minimizers as the original problem with orthogonality constraints. Based on PenC, we first propose a first-order algorithm called PenCF and establish its global convergence and local linear convergence rate under some mild assumptions. For the case that the computation and storage of Hessian is achievable, and we pursue high precision solution and fast local convergence rate, a second-order approach called PenCS is proposed for solving PenC. To avoid expensive calculation or solving a hard subproblem in computing the Newton step, we propose a new strategy to do it approximately which still leads to quadratic convergence locally. Moreover, the main iterations of both PenCF and PenCS are orthonormalization-free and hence parallelizable. Numerical experiments illustrate that PenCF is comparable with the existing first-order methods. Furthermore, PenCS shows its stability and high efficiency in obtaining high precision solution comparing with the existing second-order methods.</span>

A Theoretical Analysis on Efficiency of Some Newton-PCG Methods

A Relative Feasibility Degree Based Approach for Constrained Optimization Problems.

Inexact Proximal Cubic Regularized Newton Methods for Convex Optimization

Inexact Newton Method for Solving Generalized Nash Equilibrium Problems

Smale's α-theory for inexact Newton methods under the γ-condition

A Comparative Study on Four Unconstrained Optimization Algorithms

Inexact Newton-type Methods for Optimisation with Nonnegativity Constraints

iNALM: An inexact Newton Augmented Lagrangian Method for Zero-One Composite Optimization

An Efficient Inexact Newton-CG Algorithm for the Smallest Enclosing Ball Problem of Large Dimensions

Nonlinear conjugate gradient methods: worst-case convergence rates via computer-assisted analyses

A class of smooth exact penalty function methods for optimization problems with orthogonality constraints

Generalized Newton Method with Positive Definite Regularization for Nonsmooth Optimization Problems with Nonisolated Solutions

An efficient semismooth Newton-AMG-based inexact primal-dual algorithm for generalized transport problems

A Note on Inexact Condition for Cubic Regularized Newton's Method

Inexact proximal DC Newton-type method for nonconvex composite functions

A Newton-CG Augmented Lagrangian Method for Semidefinite Programming

On the local convergence of the semismooth Newton method for composite optimization

Comprehensive Pareto Efficiency in Robust Counterpart Optimization

Decentralized Approximate Newton Methods for Convex Optimization on Networked Systems

Branch-and-Bound Performance Estimation Programming: A Unified Methodology for Constructing Optimal Optimization Methods

Quasi-Newton approaches to Interior Point Methods for quadratic problems