Abstract:To construct a parallel approach for solving optimization problems with orthogonality constraints is usually regarded as an extremely difficult mission, due to the low scalability of the orthonormalization procedure. However, such a demand is particularly huge in some application areas such as materials computation. In this paper, we propose a proximal linearized augmented Lagrangian algorithm (PLAM) for solving optimization problems with orthogonality constraints. Unlike the classical augmented Lagrangian methods, in our algorithm, the prime variables are updated by minimizing a proximal linearized approximation of the augmented Lagrangian function; meanwhile the dual variables are updated by a closed-form expression which holds at any first-order stationary point. The orthonormalization procedure is only invoked once at the last step of the above-mentioned algorithm if high-precision feasibility is needed. Consequently, the main parts of the proposed algorithm can be parallelized naturally. We establish global subsequence convergence, worst-case complexity, and local convergence rate for PLAM under some mild assumptions. To reduce the sensitivity of the penalty parameter, we put forward a modification of PLAM, which is called parallelizable columnwise block minimization of PLAM (PCAL). Numerical experiments in serial illustrate that the novel updating rule for the Lagrangian multipliers significantly accelerates the convergence of PLAM and makes it comparable with the existent feasible solvers for optimization problems with orthogonality constraints, and the performance of PCAL does not highly rely on the choice of the penalty parameter. Numerical experiments under parallel environment demonstrate that PCAL attains good performance and high scalability in solving discretized Kohn--Sham total energy minimization problems.

Generalized Left-Localized Cayley Parametrization for Optimization with Orthogonality Constraints

Adaptive Localized Cayley Parametrization for Optimization Over Stiefel Manifold and Its Convergence Rate Analysis

A Quasi-Sequential Algorithm for PDE-constrained Optimization Based on Space–time Orthogonal Collocation on Finite Elements

Direct Approaches for PDE-constrained Dynamic Optimization Based on Space–time Orthogonal Collocation on Finite Elements

A Feasible Method for Optimization with Orthogonality Constraints

Global Optimization with Orthogonality Constraints Via Stochastic Diffusion on Manifold

Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

Optimization over convex polyhedra via Hadamard parametrizations

Optimization without Retraction on the Random Generalized Stiefel Manifold

A Riemannian optimization method on the indefinite Stiefel manifold

Stable and Total Fenchel Duality for Convex Optimization Problems in Locally Convex Spaces

The effect of smooth parametrizations on nonconvex optimization landscapes

Quadratic Optimization with Orthogonality Constraint: Explicit Łojasiewicz Exponent and Linear Convergence of Retraction-Based Line-Search and Stochastic Variance-Reduced Gradient Methods

Manifold-Constrained Geometric Optimization via Local Parameterizations

A Theory of the NEPv Approach for Optimization On the Stiefel Manifold

Sequential subspace methods on Stiefel manifold optimization problems

Parallelizable Algorithms for Optimization Problems with Orthogonality Constraints

New vector transport operators extending a Riemannian CG algorithm to generalized Stiefel manifold with low-rank applications

Orthogonal Directions Constrained Gradient Method: from non-linear equality constraints to Stiefel manifold

Multipliers Correction Methods for Optimization Problems over the Stiefel Manifold