Abstract:In this paper, we propose a novel extra-gradient difference acceleration algorithm for solving constrained nonconvex-nonconcave (NC-NC) minimax problems. In particular, we design a new extra-gradient difference step to obtain an important quasi-cocoercivity property, which plays a key role to significantly improve the convergence rate in the constrained NC-NC setting without additional structural assumption. Then momentum acceleration is also introduced into our dual accelerating update step. Moreover, we prove that, to find an $\epsilon$-stationary point of the function $f$, our algorithm attains the complexity $\mathcal{O}(\epsilon^{-2})$ in the constrained NC-NC setting, while the best-known complexity bound is $\widetilde{\mathcal{O}}(\epsilon^{-4})$, where $\widetilde{\mathcal{O}}(\cdot)$ hides logarithmic factors compared to $\mathcal{O}(\cdot)$. As the special cases of the constrained NC-NC setting, our algorithm can also obtain the same complexity $\mathcal{O}(\epsilon^{-2})$ for both the nonconvex-concave (NC-C) and convex-nonconcave (C-NC) cases, while the best-known complexity bounds are $\widetilde{\mathcal{O}}(\epsilon^{-2.5})$ for the NC-C case and $\widetilde{\mathcal{O}}(\epsilon^{-4})$ for the C-NC case. For fair comparison with existing algorithms, we also analyze the complexity bound to find $\epsilon$-stationary point of the primal function $\phi$ for the constrained NC-C problem, which shows that our algorithm can improve the complexity bound from $\widetilde{\mathcal{O}}(\epsilon^{-3})$ to $\mathcal{O}(\epsilon^{-2})$. To the best of our knowledge, this is the first time that the proposed algorithm improves the best-known complexity bounds from $\mathcal{O}(\epsilon^{-4})$ and $\widetilde{\mathcal{O}}(\epsilon^{-3})$ to $\mathcal{O}(\epsilon^{-2})$ in both the NC-NC and NC-C settings.

Practical Acceleration of the Condat-Vũ Algorithm

Practical Acceleration of the Condat–Vũ Algorithm

Accelerated Primal-Dual Algorithms for Distributed Smooth Convex Optimization over Networks

Acceleration of Primal–Dual Methods by Preconditioning and Simple Subproblem Procedures

Restarted Primal-Dual Hybrid Conjugate Gradient Method for Large-Scale Quadratic Programming

Accelerated nonlinear primal-dual hybrid gradient methods with applications to supervised machine learning

Transformed Primal-Dual Methods with Variable-Preconditioners

Accelerating preconditioned ADMM via degenerate proximal point mappings

Accelerated Proximal Coordinate Gradient Methods

Accelerated Primal-Dual Gradient Method for Smooth and Convex-Concave Saddle-Point Problems with Bilinear Coupling

Monitoring the Convergence Speed of PDHG to Find Better Primal and Dual Step Sizes

Combining Conjugate Gradient and Momentum for Unconstrained Stochastic Optimization With Applications to Machine Learning

Accelerated Primal-Dual Projection Neurodynamic Approach With Time Scaling for Linear and Set Constrained Convex Optimization Problems

Accelerated Primal-Dual Proximal Gradient Splitting Methods for Convex-Concave Saddle-Point Problems

Convergence Rates of Accelerated Markov Gradient Descent with Applications in Reinforcement Learning

Accelerated Algorithms for Constrained Convex Optimization

Distributed Accelerated Proximal Coordinate Gradient Methods

A Single-Loop Accelerated Extra-Gradient Difference Algorithm with Improved Complexity Bounds for Constrained Minimax Optimization.

Breaking the Convergence Barrier: Optimization via Fixed-Time Convergent Flows

Make Workers Work Harder: Decoupled Asynchronous Proximal Stochastic Gradient Descent.

Enhancing Stochastic Gradient Descent: A Unified Framework and Novel Acceleration Methods for Faster Convergence