Abstract:SIAM Journal on Optimization, Volume 34, Issue 2, Page 1679-1707, June 2024. Classical primal-dual algorithms attempt to solve [math] by alternately minimizing over the primal variable [math] through primal descent and maximizing the dual variable [math] through dual ascent. However, when [math] is highly nonconvex with complex constraints in [math], the minimization over [math] may not achieve global optimality and, hence, the dual ascent step loses its valid intuition. This observation motivates us to propose a new class of primal-dual algorithms for nonconvex constrained optimization with the key feature to reverse dual ascent to a conceptually new dual descent, in a sense, elevating the dual variable to the same status as the primal variable. Surprisingly, this new dual scheme achieves some best iteration complexities for solving nonconvex optimization problems. In particular, when the dual descent step is scaled by a fractional constant, we name it scaled dual descent (SDD), otherwise, unscaled dual descent (UDD). For nonconvex multiblock optimization with nonlinear equality constraints, we propose SDD-alternating direction method of multipliers (SDD-ADMM) and show that it finds an [math]-stationary solution in [math] iterations. The complexity is further improved to [math] and [math] under proper conditions. We also propose UDD-augmented Lagrangian method (UDD-ALM), combining UDD with ALM, for weakly convex minimization over affine constraints. We show that UDD-ALM finds an [math]-stationary solution in [math] iterations. These complexity bounds for both algorithms either achieve or improve the best-known results in the ADMM and ALM literature. Moreover, SDD-ADMM addresses a long-standing limitation of existing ADMM frameworks.

A Class of Simple Dual Nonlinear Methods for Solving Linear Inequalities

An Efficient Interior-Point Algorithm with New Non-Monotone Line Search Filter Method for Nonlinear Constrained Programming

Unconstrained Optimization Method for Solving Linear Inequalities

A Parallel Method With Hybrid Algorithms For Mixed Integer Nonlinear Programming

New Dual Formulations In Constrained Integer Programming

Dynamic system methods for solving mixed linear matrix inequalities and linear vector inequalities and equalities

SDPNAL+: A Matlab software for semidefinite programming with bound constraints (version 1.0)

A QP-FREE AND SUPERLINEARLY CONVERGENT ALGORITHM FOR INEQUALITY CONSTRAINEDOPTIMIZATIONS

A primal-dual interior-point relaxation method with global and rapidly local convergence for nonlinear programs

SDP Duals without Duality Gaps for a Class of Convex Minimax Programs

On Solving a Class of Linear Semi-Infinite Programs by the Trigonometric Moment

A version of the simplex method for solving linear systems of inequalities and linear programming problems

Dual Descent Augmented Lagrangian Method and Alternating Direction Method of Multipliers

A Nonlinear Lagrangian Dual For Integer Programming

NE/SQP: A robust algorithm for the nonlinear complementarity problem

Affine-gradient algorithm for solving linear inequalities

An efficient algorithm for nonlinear integer programming problems arising in series–parallel reliability systems

An Overview and Comparison of Spectral Bundle Methods for Primal and Dual Semidefinite Programs

A Hybrid Genetic Algorithm for Solving Nonlinear Programming Problems with Inequality Constraints

Linearly Convergent First-Order Algorithms for Semi-definite Programming

A Finite Newton Algorithm for Non-degenerate Piecewise Linear Systems.