Abstract:Much recent research effort has been directed to the development of efficient algorithms for solving minimax problems with theoretical convergence guarantees due to the relevance of these problems to a few emergent applications. In this paper, we propose a unified single-loop alternating gradient projection (AGP) algorithm for solving smooth nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. AGP employs simple gradient projection steps for updating the primal and dual variables alternatively at each iteration. We show that it can find an -stationary point of the objective function in (resp. ) iterations under nonconvex-strongly concave (resp. nonconvex–concave) setting. Moreover, its gradient complexity to obtain an -stationary point of the objective function is bounded by (resp., ) under the strongly convex–nonconcave (resp., convex–nonconcave) setting. To the best of our knowledge, this is the first time that a simple and unified single-loop algorithm is developed for solving both nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. Moreover, the complexity results for solving the latter (strongly) convex–nonconcave minimax problems have never been obtained before in the literature. Numerical results show the efficiency of the proposed AGP algorithm. Furthermore, we extend the AGP algorithm by presenting a block alternating proximal gradient (BAPG) algorithm for solving more general multi-block nonsmooth nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. We can similarly establish the gradient complexity of the proposed algorithm under these four different settings.

More Efficient Accelerated Proximal Algorithm for Nonconvex Problems

Efficient Inexact Proximal Gradient Algorithm for Nonconvex Problems.

Accelerated Proximal Gradient Methods for Nonconvex Programming

A class of modified accelerated proximal gradient methods for nonsmooth and nonconvex minimization problems

Accelerated Proximal Coordinate Gradient Methods

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

A nonmonotone accelerated proximal gradient method with variable stepsize strategy for nonsmooth and nonconvex minimization problems

Distributed Accelerated Proximal Coordinate Gradient Methods

Accelerated Quasi-Newton Proximal Extragradient: Faster Rate for Smooth Convex Optimization

A unified single-loop alternating gradient projection algorithm for nonconvex–concave and convex–nonconcave minimax problems

Accelerated Bregman Proximal Gradient Methods for Relatively Smooth Convex Optimization

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Smoothing Accelerated Proximal Gradient Method with Fast Convergence Rate for Nonsmooth Convex Optimization Beyond Differentiability

An adaptive proximal point algorithm framework and application to large-scale optimization

The convergence rate of the accelerated proximal gradient algorithm for Multiobjective Optimization is faster than $O(1/k^2)$

Accelerated gradient methods for nonconvex nonlinear and stochastic programming

Asynchronous Stochastic Proximal Methods for Nonconvex Nonsmooth Optimization.

Inexact Proximal Gradient Methods for Non-convex and Non-smooth Optimization

A Simple Proximal Stochastic Gradient Method for Nonsmooth Nonconvex Optimization.

Accelerated Algorithms for Nonconvex Optimization

Adaptive Proximal Average Approximation for Composite Convex Minimization.