Abstract:Using convex combination and linesearch techniques, we introduce a novel primal-dual algorithm for solving structured convex-concave saddle point problems with a generic smooth nonbilinear coupling term. Our adaptive linesearch strategy works under specific local smoothness conditions, allowing for potentially larger stepsizes. For an important class of structured convex optimization problems, the proposed algorithm reduces to a fully adaptive proximal gradient algorithm without linesearch, thereby representing an advancement over the golden ratio algorithm delineated in [Y. Malitsky, Math. Program. 2020]. We establish global pointwise and ergodic sublinear convergence rate of the algorithm measured by the primal-dual gap function in the general case. When the coupling term is linear in the dual variable, we measure the convergence rate by function value residual and constraint violation of an equivalent constrained optimization problem. Furthermore, an accelerated algorithm achieving the faster O(1/N^2) ergodic convergence rate is presented for the strongly convex case, where N denotes the iteration number. Our numerical experiments on quadratically constrained quadratic programming and sparse logistic regression problems indicate the new algorithm is significantly faster than the comparison algorithms.

The Saddle-Gradient Method for Linear Constrained Convex Programming

A GLOBALLY AND SUPERLIEARLY CONVERGENT FEASIBLE SQP ALGORITHM FOR DEGENERATE CONSTRAINED OPTIMIZATION

An Efficient Interior-Point Algorithm with New Non-Monotone Line Search Filter Method for Nonlinear Constrained Programming

A convex combination based primal-dual algorithm with linesearch for general convex-concave saddle point problems

Accelerated Primal-Dual Gradient Method for Smooth and Convex-Concave Saddle-Point Problems with Bilinear Coupling

A Theoretical Revisit to Linear Convergence for Saddle Point Problems

A Universal Accelerated Primal–Dual Method for Convex Optimization Problems

Linear Convergence of Subgradient Algorithm for Convex Feasibility on Riemannian Manifolds

A globally convergent SQP-type method with least constraint violation for nonlinear semidefinite programming

Convergence rate analysis of the gradient descent–ascent method for convex–concave saddle-point problems

Existence of a Saddle Point in Nonconvex Constrained Optimization

A Primal Majorized Semismooth Newton-CG Augmented Lagrangian Method for Large-Scale Linearly Constrained Convex Programming

GLOBAL OPTIMIZATION FOR NON-CONVEX PROGRAMS VIA CONVEX PROXIMAL POINT METHOD

New Dual Formulations In Constrained Integer Programming

Nonlinear preconditioned primal-dual method for a class of structured minimax problems

Faster Accelerated First-order Methods for Convex Optimization with Strongly Convex Function Constraints

Primal-dual Accelerated Mirror-Descent Method for Constrained Bilinear Saddle-Point Problems

A Proximal-Gradient Method for Constrained Optimization

Convergent Lagrangian and Contour Cut Method for Nonlinear Integer Programming with a Quadratic Objective Function

Accelerated Primal-dual Scheme for a Class of Stochastic Nonconvex-concave Saddle Point Problems