Abstract:Augmented Lagrangian method (ALM) has been popularly used for solving constrained optimization problems. Practically, subproblems for updating primal variables in the framework of ALM usually can only be solved inexactly. The convergence and local convergence speed of ALM have been extensively studied. However, the global convergence rate of the inexact ALM is still open for problems with nonlinear inequality constraints. In this paper, we work on general convex programs with both equality and inequality constraints. For these problems, we establish the global convergence rate of the inexact ALM and estimate its iteration complexity in terms of the number of gradient evaluations to produce a primal and/or primal-dual solution with a specified accuracy. We first establish an ergodic convergence rate result of the inexact ALM that uses constant penalty parameters or geometrically increasing penalty parameters. Based on the convergence rate result, we then apply Nesterov's optimal first-order method on each primal subproblem and estimate the iteration complexity of the inexact ALM. We show that if the objective is convex, then \(O(\varepsilon ^{-1})\) gradient evaluations are sufficient to guarantee a primal \(\varepsilon \)-solution in terms of both primal objective and feasibility violation. If the objective is strongly convex, the result can be improved to \(O(\varepsilon ^{-\frac{1}{2}}|\log \varepsilon |)\). To produce a primal-dual \(\varepsilon \)-solution, more gradient evaluations are needed for convex case, and the number is \(O(\varepsilon ^{-\frac{4}{3}})\), while for strongly convex case, the number is still \(O(\varepsilon ^{-\frac{1}{2}}|\log \varepsilon |)\). Finally, we establish a nonergodic convergence rate result of the inexact ALM that uses geometrically increasing penalty parameters. This result is established only for the primal problem. We show that the nonergodic iteration complexity result is in the same order as that for the ergodic result. Numerical experiments on quadratically constrained quadratic programming are conducted to compare the performance of the inexact ALM with different settings.

An Inexact Primal-Dual Algorithm for Semi-Infinite Programming

Convex semi-infinite programming algorithms with inexact separation oracles

An Inexact Primal-Dual Smoothing Framework for Large-Scale Non-Bilinear Saddle Point Problems

An inexact golden ratio primal-dual algorithm with linesearch step for a saddle point problem

Corporative Stochastic Approximation with Random Constraint Sampling for Semi-Infinite Programming

An efficient symmetric primal-dual algorithmic framework for saddle point problems

Inexact cuts for Deterministic and Stochastic Dual Dynamic Programming applied to convex nonlinear optimization problems

O C ] 2 1 D ec 2 01 8 Corporative Stochastic Approximation with Random Constraint Sampling for Semi-Infinite Programming

A new primal-dual algorithm for structured convex optimization involving a lipschitzian term

An Interior Point-Proximal Method of Multipliers for Positive Semi-Definite Programming

Convergence analysis of approximate primal solutions in dual first-order methods

IPRSDP: a primal-dual interior-point relaxation algorithm for semidefinite programming

Gradient Methods with Dynamic Inexact Oracles

Inexact Augmented Lagrangian Methods for Conic Programs: Quadratic Growth and Linear Convergence

Convergence to a second-order critical point by a primal-dual interior point trust-region method for nonlinear semidefinite programming

A Primal-Dual Infeasible Interior Point Algorithm for Semidefinite Programming

Inexact cuts in Deterministic and Stochastic Dual Dynamic Programming applied to linear optimization problems

Convergence analysis of a primal-dual optimization-by-continuation algorithm

Iteration complexity of inexact augmented Lagrangian methods for constrained convex programming

A New Primal-Dual Algorithm for a Class of Nonlinear Compositional Convex Optimization Problems