Abstract:Gradient-based algorithms have shown great promise in solving large (two-player) zero-sum games. However, their success has been mostly confined to the low-precision regime since the number of iterations grows polynomially in $1/\epsilon$, where $\epsilon > 0$ is the duality gap. While it has been well-documented that linear convergence -- an iteration complexity scaling as $\textsf{log}(1/\epsilon)$ -- can be attained even with gradient-based algorithms, that comes at the cost of introducing a dependency on certain condition number-like quantities which can be exponentially large in the description of the game. To address this shortcoming, we examine the iteration complexity of several gradient-based algorithms in the celebrated framework of smoothed analysis, and we show that they have polynomial smoothed complexity, in that their number of iterations grows as a polynomial in the dimensions of the game, $\textsf{log}(1/\epsilon)$, and $1/\sigma$, where $\sigma$ measures the magnitude of the smoothing perturbation. Our result applies to optimistic gradient and extra-gradient descent/ascent, as well as a certain iterative variant of Nesterov's smoothing technique. From a technical standpoint, the proof proceeds by characterizing and performing a smoothed analysis of a certain error bound, the key ingredient driving linear convergence in zero-sum games. En route, our characterization also makes a natural connection between the convergence rate of such algorithms and perturbation-stability properties of the equilibrium, which is of interest beyond the model of smoothed complexity.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the convergence problem faced by the gradient descent algorithm in high - precision solutions in zero - sum games. Specifically: 1. **Limitations of existing methods**: - The gradient descent algorithm performs well in low - precision solutions (i.e., a large dual gap $\epsilon$), but when a high - precision solution (i.e., a very small $\epsilon$) is required, the number of iterations will increase polynomially with $1/\epsilon$, which makes the computational complexity unacceptable. - Although the linear convergence rate (i.e., the iteration complexity is $\log(1/\epsilon)$) can be achieved by introducing certain condition numbers, these condition numbers may increase exponentially with the game description, resulting in difficulties in actual calculations. 2. **Research objectives**: - To overcome the above problems, the authors studied the iteration complexity of several gradient descent algorithms in zero - sum games under the smoothed analysis framework. They hope to prove that these algorithms can still converge to high - precision solutions with polynomial complexity under the game matrix after a small random perturbation. 3. **Specific contributions**: - The authors showed that the optimistic gradient descent/ascent (OGDA), the extra - gradient descent/ascent (EGDA), and the iterative variant of Nesterov smoothing technique (IterSmooth) have polynomial smoothing complexity with high probability, that is, they can converge to the $\epsilon$-equilibrium point within $\text{poly}(n, m, 1/\sigma)\cdot\log(1/\epsilon)$ iterations. - For the optimistic multiplicative weight update (OMWU), although it does not reach polynomial complexity, it has a significant improvement compared to the worst - case analysis. 4. **Key tools**: - The authors introduced an error bound and proved through the smoothed analysis framework that the modulus $\kappa$ of this error bound is a polynomial function of $\sigma$ and $1/(nm)$ with high probability. - In this way, they linked the convergence of the gradient descent algorithm with the stability of the game, thus providing a new perspective for understanding the performance of these algorithms in practical applications. In summary, this paper aims to solve the convergence problem of the gradient descent algorithm in high - precision solutions of zero - sum games by introducing the smoothed analysis framework and provides a new theoretical tool to evaluate the performance of these algorithms.

Convergence of $\text{log}(1/ε)$ for Gradient-Based Algorithms in Zero-Sum Games without the Condition Number: A Smoothed Analysis

Exponential Convergence of Gradient Methods in Concave Network Zero-sum Games

On the Last-iterate Convergence in Time-varying Zero-sum Games: Extra Gradient Succeeds where Optimism Fails

Last-Iterate Convergence of Payoff-Based Independent Learning in Zero-Sum Stochastic Games

Accelerated zero-order SGD under high-order smoothness and overparameterized regime

Fast Last-Iterate Convergence of Learning in Games Requires Forgetful Algorithms

On the Convergence of No-Regret Learning Dynamics in Time-Varying Games

On Gradient-Based Learning in Continuous Games

Zero-Gradient-Sum Algorithms for Distributed Convex Optimization: The Continuous-Time Case

Learning Zero-Sum Linear Quadratic Games with Improved Sample Complexity and Last-Iterate Convergence

Gradient-free optimization of highly smooth functions: improved analysis and a new algorithm

Optimizing $(L_0, L_1)$-Smooth Functions by Gradient Methods

Directional Smoothness and Gradient Methods: Convergence and Adaptivity

Convergence of Learning Dynamics in Stackelberg Games

Convergence Analysis of Adaptive Gradient Methods under Refined Smoothness and Noise Assumptions

Generalized-Smooth Nonconvex Optimization is As Efficient As Smooth Nonconvex Optimization

Global Convergence of Langevin Dynamics Based Algorithms for Nonconvex Optimization

Convergence analysis and acceleration of the smoothing methods for solving extensive-form games

Towards convergence to Nash equilibria in two-team zero-sum games

Local convergence of simultaneous min-max algorithms to differential equilibrium on Riemannian manifold

A Whole New Ball Game: A Primal Accelerated Method for Matrix Games and Minimizing the Maximum of Smooth Functions