Abstract:Much recent research effort has been directed to the development of efficient algorithms for solving minimax problems with theoretical convergence guarantees due to the relevance of these problems to a few emergent applications. In this paper, we propose a unified single-loop alternating gradient projection (AGP) algorithm for solving smooth nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. AGP employs simple gradient projection steps for updating the primal and dual variables alternatively at each iteration. We show that it can find an -stationary point of the objective function in (resp. ) iterations under nonconvex-strongly concave (resp. nonconvex–concave) setting. Moreover, its gradient complexity to obtain an -stationary point of the objective function is bounded by (resp., ) under the strongly convex–nonconcave (resp., convex–nonconcave) setting. To the best of our knowledge, this is the first time that a simple and unified single-loop algorithm is developed for solving both nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. Moreover, the complexity results for solving the latter (strongly) convex–nonconcave minimax problems have never been obtained before in the literature. Numerical results show the efficiency of the proposed AGP algorithm. Furthermore, we extend the AGP algorithm by presenting a block alternating proximal gradient (BAPG) algorithm for solving more general multi-block nonsmooth nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. We can similarly establish the gradient complexity of the proposed algorithm under these four different settings.

GLOBAL OPTIMIZATION FOR NON-CONVEX PROGRAMS VIA CONVEX PROXIMAL POINT METHOD

A GLOBALLY AND SUPERLIEARLY CONVERGENT FEASIBLE SQP ALGORITHM FOR DEGENERATE CONSTRAINED OPTIMIZATION

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

Stochastic Successive Convex Approximation for Non-Convex Constrained Stochastic Optimization

A Symplectic Discretization Based Proximal Point Algorithm for Convex Minimization

An adaptive proximal point algorithm framework and application to large-scale optimization

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

A Feasible Level Proximal Point Method for Nonconvex Sparse Constrained Optimization

A Decentralized Proximal Point-type Method for Non-convex Non-concave Saddle Point Problems

A globally convergent proximal Newton-type method in nonsmooth convex optimization

A New Global Optimization Method Based on Simplex Branching for Solving a Class of Non-Convex QCQP Problems

A Proximal-Gradient Method for Constrained Optimization

Directional proximal point method for convex optimization

Globally Optimal Solutions to a Class of Fractional Optimization Problems Based on Proximal Gradient Algorithm

Proximal Projection Method for Stable Linearly Constrained Optimization

A unified single-loop alternating gradient projection algorithm for nonconvex–concave and convex–nonconcave minimax problems

Smoothed Proximal Lagrangian Method for Nonlinear Constrained Programs

A globally convergent difference-of-convex algorithmic framework and application to log-determinant optimization problems

Accelerated Proximal Gradient Methods for Nonconvex Programming

A Proximal Point Algorithm For Log-Determinant Optimization With Group Lasso Regularization