Abstract:To construct a parallel approach for solving optimization problems with orthogonality constraints is usually regarded as an extremely difficult mission, due to the low scalability of the orthonormalization procedure. However, such a demand is particularly huge in some application areas such as materials computation. In this paper, we propose a proximal linearized augmented Lagrangian algorithm (PLAM) for solving optimization problems with orthogonality constraints. Unlike the classical augmented Lagrangian methods, in our algorithm, the prime variables are updated by minimizing a proximal linearized approximation of the augmented Lagrangian function; meanwhile the dual variables are updated by a closed-form expression which holds at any first-order stationary point. The orthonormalization procedure is only invoked once at the last step of the above-mentioned algorithm if high-precision feasibility is needed. Consequently, the main parts of the proposed algorithm can be parallelized naturally. We establish global subsequence convergence, worst-case complexity, and local convergence rate for PLAM under some mild assumptions. To reduce the sensitivity of the penalty parameter, we put forward a modification of PLAM, which is called parallelizable columnwise block minimization of PLAM (PCAL). Numerical experiments in serial illustrate that the novel updating rule for the Lagrangian multipliers significantly accelerates the convergence of PLAM and makes it comparable with the existent feasible solvers for optimization problems with orthogonality constraints, and the performance of PCAL does not highly rely on the choice of the penalty parameter. Numerical experiments under parallel environment demonstrate that PCAL attains good performance and high scalability in solving discretized Kohn--Sham total energy minimization problems.

Nonconvex and Nonsmooth Optimization with Generalized Orthogonality Constraints: an Approximate Augmented Lagrangian Method

Parallelizable Algorithms for Optimization Problems with Orthogonality Constraints

Implicit augmented Lagrangian and generalized optimization

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

A class of smooth exact penalty function methods for optimization problems with orthogonality constraints

An Exact Penalty Approach for Optimization with Nonnegative Orthogonality Constraints

The inexact power augmented Lagrangian method for constrained nonconvex optimization

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

Global Convergence Analysis of the Power Proximal Point and Augmented Lagrangian Method

An Inexact Augmented Lagrangian Framework for Nonconvex Optimization with Nonlinear Constraints

A Proximal-Gradient Method for Constrained Optimization

Damped Proximal Augmented Lagrangian Method for weakly-Convex Problems with Convex Constraints

Error bound and exact penalty method for optimization problems with nonnegative orthogonal constraint

Augmented Lagrangian optimization under fixed-point arithmetic

A Semismooth Newton Based Augmented Lagrangian Method for Nonsmooth Optimization on Matrix Manifolds

Penalty Methods for a Class of Non-Lipschitz Optimization Problems

Smoothed Proximal Lagrangian Method for Nonlinear Constrained Programs

An Augmented Lagrangian Method for State Constrained Linear Parabolic Optimal Control Problems

A Penalty-Free Infeasible Approach for a Class of Nonsmooth Optimization Problems Over the Stiefel Manifold

Inexact Augmented Lagrangian Methods for Conic Programs: Quadratic Growth and Linear Convergence

A nonsmooth exact penalty method for equality-constrained optimization: complexity and implementation