Abstract:To construct a parallel approach for solving optimization problems with orthogonality constraints is usually regarded as an extremely difficult mission, due to the low scalability of the orthonormalization procedure. However, such a demand is particularly huge in some application areas such as materials computation. In this paper, we propose a proximal linearized augmented Lagrangian algorithm (PLAM) for solving optimization problems with orthogonality constraints. Unlike the classical augmented Lagrangian methods, in our algorithm, the prime variables are updated by minimizing a proximal linearized approximation of the augmented Lagrangian function; meanwhile the dual variables are updated by a closed-form expression which holds at any first-order stationary point. The orthonormalization procedure is only invoked once at the last step of the above-mentioned algorithm if high-precision feasibility is needed. Consequently, the main parts of the proposed algorithm can be parallelized naturally. We establish global subsequence convergence, worst-case complexity, and local convergence rate for PLAM under some mild assumptions. To reduce the sensitivity of the penalty parameter, we put forward a modification of PLAM, which is called parallelizable columnwise block minimization of PLAM (PCAL). Numerical experiments in serial illustrate that the novel updating rule for the Lagrangian multipliers significantly accelerates the convergence of PLAM and makes it comparable with the existent feasible solvers for optimization problems with orthogonality constraints, and the performance of PCAL does not highly rely on the choice of the penalty parameter. Numerical experiments under parallel environment demonstrate that PCAL attains good performance and high scalability in solving discretized Kohn--Sham total energy minimization problems.

Quadratic Optimization with Orthogonality Constraint: Explicit Łojasiewicz Exponent and Linear Convergence of Retraction-Based Line-Search and Stochastic Variance-Reduced Gradient Methods

Global Optimization with Orthogonality Constraints Via Stochastic Diffusion on Manifold

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

A Stochastic Sequential Quadratic Optimization Algorithm for Nonlinear Equality Constrained Optimization with Rank-Deficient Jacobians

A Feasible Method for Optimization with Orthogonality Constraints

Modified Line Search Sequential Quadratic Methods for Equality-Constrained Optimization with Unified Global and Local Convergence Guarantees

Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

Structured Quasi-Newton Methods for Optimization with Orthogonality Constraints.

Global Convergence of the Polak-Ribière-Polyak Conjugate Gradient Method with an Armijo-type Inexact Line Search for Nonconvex Unconstrained Optimization Problems

Parallelizable Algorithms for Optimization Problems with Orthogonality Constraints

Nonconvex and Nonsmooth Optimization with Generalized Orthogonality Constraints: an Approximate Augmented Lagrangian Method

Orthogonal Directions Constrained Gradient Method: from non-linear equality constraints to Stiefel manifold

Sequential Quadratic Optimization for Stochastic Optimization with Deterministic Nonlinear Inequality and Equality Constraints

A block Lanczos method for large-scale quadratic minimization problems with orthogonality constraints

Linear Convergence of Stochastic Iterative Greedy Algorithms With Sparse Constraints

Stochastic Optimization for Nonconvex Problem with Inexact Hessian Matrix, Gradient, and Function

Weakly Convex Optimization over Stiefel Manifold Using Riemannian Subgradient-Type Methods

Eigen-componentwise convergence of SGD on quadratic programming

Infeasible Deterministic, Stochastic, and Variance-Reduction Algorithms for Optimization under Orthogonality Constraints

Stochastic Optimization for Non-convex Problem with Inexact Hessian Matrix, Gradient, and Function

A Stochastic Inexact Sequential Quadratic Optimization Algorithm for Nonlinear Equality-Constrained Optimization