Abstract:Much recent research effort has been directed to the development of efficient algorithms for solving minimax problems with theoretical convergence guarantees due to the relevance of these problems to a few emergent applications. In this paper, we propose a unified single-loop alternating gradient projection (AGP) algorithm for solving smooth nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. AGP employs simple gradient projection steps for updating the primal and dual variables alternatively at each iteration. We show that it can find an -stationary point of the objective function in (resp. ) iterations under nonconvex-strongly concave (resp. nonconvex–concave) setting. Moreover, its gradient complexity to obtain an -stationary point of the objective function is bounded by (resp., ) under the strongly convex–nonconcave (resp., convex–nonconcave) setting. To the best of our knowledge, this is the first time that a simple and unified single-loop algorithm is developed for solving both nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. Moreover, the complexity results for solving the latter (strongly) convex–nonconcave minimax problems have never been obtained before in the literature. Numerical results show the efficiency of the proposed AGP algorithm. Furthermore, we extend the AGP algorithm by presenting a block alternating proximal gradient (BAPG) algorithm for solving more general multi-block nonsmooth nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. We can similarly establish the gradient complexity of the proposed algorithm under these four different settings.

Derivative-Free Alternating Projection Algorithms for General Nonconvex-Concave Minimax Problems

Derivative-free Alternating Projection Algorithms for General Nonconvex-Concave Minimax Problems

Zeroth-Order primal-dual Alternating Projection Gradient Algorithms for Nonconvex Minimax Problems with Coupled linear Constraints

Two Completely Parameter-Free Alternating Gradient Projection Algorithms for Nonconvex-(strongly) Concave Minimax Problems

A unified single-loop alternating gradient projection algorithm for nonconvex–concave and convex–nonconcave minimax problems

Primal Dual Alternating Proximal Gradient Algorithms for Nonsmooth Nonconvex Minimax Problems with Coupled Linear Constraints

Obtaining Lower Query Complexities Through Lightweight Zeroth-Order Proximal Gradient Algorithms

Zeroth-order Random Subspace Algorithm for Non-smooth Convex Optimization

A Riemannian Alternating Descent Ascent Algorithmic Framework for Nonconvex-Linear Minimax Problems on Riemannian Manifolds

O(log T) Projections for Stochastic Optimization of Smooth and Strongly Convex Functions

Accelerated Proximal Gradient Methods for Nonconvex Programming

Complexities in Projection-Free Stochastic Non-convex Minimization

Distributed Zeroth-Order Optimization: Convergence Rates That Match Centralized Counterpart

Alternating and Parallel Proximal Gradient Methods for Nonsmooth, Nonconvex Minimax: A Unified Convergence Analysis

Nonconvex Zeroth-Order Stochastic ADMM Methods with Lower Function Query Complexity

An approximation proximal gradient algorithm for nonconvex-linear minimax problems with nonconvex nonsmooth terms

Projection-Free Non-Smooth Convex Programming

Adaptive Alternating Minimization Algorithms

An Infeasible-Point Subgradient Method Using Adaptive Approximate Projections

New nonasymptotic convergence rates of stochastic proximal pointalgorithm for convex optimization problems