Abstract:Augmented Lagrangian method (ALM) has been popularly used for solving constrained optimization problems. Practically, subproblems for updating primal variables in the framework of ALM usually can only be solved inexactly. The convergence and local convergence speed of ALM have been extensively studied. However, the global convergence rate of the inexact ALM is still open for problems with nonlinear inequality constraints. In this paper, we work on general convex programs with both equality and inequality constraints. For these problems, we establish the global convergence rate of the inexact ALM and estimate its iteration complexity in terms of the number of gradient evaluations to produce a primal and/or primal-dual solution with a specified accuracy. We first establish an ergodic convergence rate result of the inexact ALM that uses constant penalty parameters or geometrically increasing penalty parameters. Based on the convergence rate result, we then apply Nesterov's optimal first-order method on each primal subproblem and estimate the iteration complexity of the inexact ALM. We show that if the objective is convex, then \(O(\varepsilon ^{-1})\) gradient evaluations are sufficient to guarantee a primal \(\varepsilon \)-solution in terms of both primal objective and feasibility violation. If the objective is strongly convex, the result can be improved to \(O(\varepsilon ^{-\frac{1}{2}}|\log \varepsilon |)\). To produce a primal-dual \(\varepsilon \)-solution, more gradient evaluations are needed for convex case, and the number is \(O(\varepsilon ^{-\frac{4}{3}})\), while for strongly convex case, the number is still \(O(\varepsilon ^{-\frac{1}{2}}|\log \varepsilon |)\). Finally, we establish a nonergodic convergence rate result of the inexact ALM that uses geometrically increasing penalty parameters. This result is established only for the primal problem. We show that the nonergodic iteration complexity result is in the same order as that for the ergodic result. Numerical experiments on quadratically constrained quadratic programming are conducted to compare the performance of the inexact ALM with different settings.

Convergence analysis of the augmented Lagrangian method for nonlinear second-order cone optimization problems

Augmented Lagrangian method for nonlinear circular conic programs: a local convergence analysis

A Unified Convergence Analysis of a Second-Order Method of Multipliers for Nonlinear Conic Programming

Local convergence of augmented Lagrangian methods for composite optimization

The convergence of conjugate gradient method with nonmonotone line search

Convergence analysis of the Gauss–Newton method for convex inclusion and convex-composite optimization problems

On the Convergence of Augmented Lagrangian Methods for Constrained Global Optimization

Inexact Augmented Lagrangian Methods for Conic Programs: Quadratic Growth and Linear Convergence

Optimality Conditions for Problems over Symmetric Cones and a Simple Augmented Lagrangian Method

Convergence analysis of primal-dual augmented Lagrangian methods and duality theory

Global Convergence Analysis of the Power Proximal Point and Augmented Lagrangian Method

Exploiting cone approximations in an augmented Lagrangian method for conic optimization

Convergence Properties of Modified and Partially-Augmented Lagrangian Methods for Mathematical Programs with Complementarity Constraints

On Saddle Points of Augmented Lagrangians for Constrained Nonconvex Optimization

The Convergence Rate and Necessary-and-Sufficient Condition for the Consistency of Isogeometric Collocation Method

convergence rates of (dual-primal) balanced augmented Lagrangian methods for linearly constrained convex programming

Second-Order Conditions for the Existence of Augmented Lagrange Multipliers for Sparse Optimization

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Iteration complexity of inexact augmented Lagrangian methods for constrained convex programming

Convergence of alternating direction method for minimizing sum of two nonconvex functions with linear constraints

The inexact power augmented Lagrangian method for constrained nonconvex optimization