M.V. Dolgopolik
Abstract:We present a so-called universal convergence theorem for inexact primal-dual penalty and augmented Lagrangian methods that can be applied to a large number of such methods and reduces their convergence analysis to verification of some simple conditions on sequences generated by these methods. If these conditions are verified, then both primal and dual convergence follow directly from the universal convergence theorem. This theorem allows one not only to derive standard convergence theorems for many existing primal-dual penalty and augmented Lagrangian methods in a unified and straightforward manner, but also to strengthen and generalize some of these theorems. In particular, we show how with the use of the universal convergence theorem one can significantly improve some existing results on convergence of a primal-dual rounded weighted $\ell_1$-penalty method, an augmented Lagrangian method for cone constrained optimization, and some other primal-dual methods.
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is how to establish a general convergence theorem for the primal - dual penalty function and the augmented Lagrangian method. Specifically, the author aims to:
1. **Reveal the basic principles of convergence analysis**: By proposing a so - called "general convergence theorem", reveal the general principles behind the convergence analysis of the primal - dual penalty function and the augmented Lagrangian method.
2. **Unify and simplify the convergence analysis**: This general convergence theorem can be applied to a large number of existing primal - dual penalty functions and augmented Lagrangian methods, and simplify their convergence analysis to verifying some simple conditions. These conditions include the infeasibility measure converging to zero, etc.
3. **Improve existing results**: Using this general convergence theorem, the author shows how to significantly improve some existing convergence results regarding the primal - dual penalty function and the augmented Lagrangian method. For example, the author shows how to improve the weighted ℓ1 - penalty function method, the augmented Lagrangian method for cone - constrained optimization problems, etc.
### Core contributions of the paper
- **General convergence theorem**: This theorem provides a framework that enables the convergence of the primal - dual method to be uniformly proven for various types of constrained optimization problems (such as nonlinear programming, cone - constrained optimization, etc.).
- **Simplify the proof process**: By introducing the general convergence theorem, the need to repeat similar argumentation processes each time a new primal - dual method is analyzed is avoided, thereby simplifying the convergence analysis.
- **Strengthen existing theories**: It is not only able to derive existing convergence theorems, but also strengthen and generalize these theorems in some cases, making their scope of application wider.
### Application examples
To demonstrate the advantages of the general convergence theorem, the author applies it to four different primal - dual penalty function/augmented Lagrangian methods:
1. **Simple primal - dual penalty function method**.
2. **Weighted ℓ1 - penalty function method**.
3. **Extended classical augmented Lagrangian method**, applicable to general cone - constrained optimization problems.
4. **Multiplier algorithm based on P - type augmented Lagrangian method**.
Through these applications, the author proves that the general convergence theorem can not only easily recover the convergence theorems of existing methods, but also significantly strengthen and generalize these theorems.
### Summary
The main objective of this paper is to provide a unified and simplified framework for the convergence analysis of the primal - dual penalty function and the augmented Lagrangian method by proposing a general convergence theorem, and improve existing results on this basis. This work helps to promote the theoretical development in the field of constrained optimization and provides a solid theoretical foundation for practical applications.