Abstract:Much recent research effort has been directed to the development of efficient algorithms for solving minimax problems with theoretical convergence guarantees due to the relevance of these problems to a few emergent applications. In this paper, we propose a unified single-loop alternating gradient projection (AGP) algorithm for solving smooth nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. AGP employs simple gradient projection steps for updating the primal and dual variables alternatively at each iteration. We show that it can find an -stationary point of the objective function in (resp. ) iterations under nonconvex-strongly concave (resp. nonconvex–concave) setting. Moreover, its gradient complexity to obtain an -stationary point of the objective function is bounded by (resp., ) under the strongly convex–nonconcave (resp., convex–nonconcave) setting. To the best of our knowledge, this is the first time that a simple and unified single-loop algorithm is developed for solving both nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. Moreover, the complexity results for solving the latter (strongly) convex–nonconcave minimax problems have never been obtained before in the literature. Numerical results show the efficiency of the proposed AGP algorithm. Furthermore, we extend the AGP algorithm by presenting a block alternating proximal gradient (BAPG) algorithm for solving more general multi-block nonsmooth nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. We can similarly establish the gradient complexity of the proposed algorithm under these four different settings.

Smoothing Projected Gradient Algorithm for a Class of Non-Lipschitz Constrained Optimization

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

Efficient Projected Gradient Methods for L 0 Constrained Optimization

Image Smoothing Via Gradient Sparsity and Surface Area Minimization

Smoothing accelerated algorithm for constrained nonsmooth convex optimization problems

O(log T) Projections for Stochastic Optimization of Smooth and Strongly Convex Functions

Efficient Projected Gradient Methods for Cardinality Constrained Optimization

A Smoothing Proximal Gradient Algorithm for Nonsmooth Convex Regression with Cardinality Penalty.

A feasible smoothing accelerated projected gradient method for nonsmooth convex optimization

A Smoothing Stochastic Gradient Method for Composite Optimization

A New Nonmonotone Spectral Projected Gradient Algorithm for Box-Constrained Optimization Problems in M×n Real Matrix Space with Application in Image Clustering.

Projected Neural Network for a Class of Non-Lipschitz Optimization Problems with Linear Constraints.

Smoothing Neural Network for Constrained Non-Lipschitz Optimization with Applications

Sign projected gradient flow: A continuous-time approach to convex optimization with linear equality constraints

A class of modified accelerated proximal gradient methods for nonsmooth and nonconvex minimization problems

Projection-Free Non-Smooth Convex Programming

Smoothing Algorithm with Edge-preserving by Extrema Constraints

A variable smoothing for Nonconvexly constrained nonsmooth optimization with application to sparse spectral clustering

Generalized Gradient Projection Neural Networks for Nonsmooth Optimization Problems

A hybrid three-term conjugate gradient projection method for constrained nonlinear monotone equations with applications

A unified single-loop alternating gradient projection algorithm for nonconvex–concave and convex–nonconcave minimax problems