Abstract:In this paper, we consider augmented Lagrangian (AL) algorithms for solving large-scale nonlinear optimization problems that execute adaptive strategies for updating the penalty parameter. Our work is motivated by the recently proposed adaptive AL trust region method by Curtis, Jiang, and Robinson [Math. Prog., DOI: <a class="link-https link-external" data-doi="10.1007/s10107-014-0784-y" href="https://doi.org/10.1007/s10107-014-0784-y" rel="external noopener nofollow">https://doi.org/10.1007/s10107-014-0784-y</a>, 2013]. The first focal point of this paper is a new variant of the approach that employs a line search rather than a trust region strategy, where a critical algorithmic feature for the line search strategy is the use of convexified piecewise quadratic models of the AL function for computing the search directions. We prove global convergence guarantees for our line search algorithm that are on par with those for the previously proposed trust region method. A second focal point of this paper is the practical performance of the line search and trust region algorithm variants in Matlab software, as well as that of an adaptive penalty parameter updating strategy incorporated into the Lancelot software. We test these methods on problems from the CUTEst and COPS collections, as well as on challenging test problems related to optimal power flow. Our numerical experience suggests that the adaptive algorithms outperform traditional AL methods in terms of efficiency and reliability. As with traditional AL algorithms, the adaptive methods are matrix-free and thus represent a viable option for solving extreme-scale problems.

A New Insight on Augmented Lagrangian Method with Applications in Machine Learning

A New Penalty Dual-Primal Augmented Lagrangian Method and Its Extensions

An Augmented Lagrangian Method for Non-Lipschitz Nonconvex Programming

A Smoothing Augmented Lagrangian Method for Nonconvex, Nonsmooth Constrained Programs and Its Applications to Bilevel Problems

Inexact Augmented Lagrangian Methods for Conic Programs: Quadratic Growth and Linear Convergence

An Augmented Lagrangian Primal-Dual Semismooth Newton Method for Multi-Block Composite Optimization

Exact Augmented Lagrangian Duality for Mixed Integer Convex Optimization

An Augmented Lagrangian Method for a Class of Inverse Quadratic Programming Problems

An Augmented Lagrangian Approach with A Variable Transformation in Nonlinear Programming

On the Convergence of Augmented Lagrangian Methods for Constrained Global Optimization

An Augmented Lagrangian Method for State Constrained Linear Parabolic Optimal Control Problems

A Unified Primal-Dual Algorithm Framework for Inequality Constrained Problems

Augmented Lagrangian optimization under fixed-point arithmetic

convergence rates of (dual-primal) balanced augmented Lagrangian methods for linearly constrained convex programming

The inexact power augmented Lagrangian method for constrained nonconvex optimization

A look at the convergence of the augmented Lagrange method for nondifferentiable convex programming from the view of a gradient method with constant stepsize

Global Convergence Analysis of the Power Proximal Point and Augmented Lagrangian Method

Adaptive Augmented Lagrangian Methods: Algorithms and Practical Numerical Experience

A First-Order Primal-Dual Method for Nonconvex Constrained Optimization Based on the Augmented Lagrangian

On Saddle Points of Augmented Lagrangians for Constrained Nonconvex Optimization

A Primal Majorized Semismooth Newton-CG Augmented Lagrangian Method for Large-Scale Linearly Constrained Convex Programming