Abstract:Learning in games is the problem where multiple players interact in a shared environment, each aiming to minimize their own regret, and it is known that an approximate equilibrium can be obtained when all players employ no-regret algorithms. Notably, by adopting optimistic follow-the-regularized-leader (OFTRL), the regret of each player after $T$ rounds is constant in two-player zero-sum games, implying that an equilibrium can be computed at a faster rate of $O(1/T)$. However, this acceleration is limited to the honest regime, where all players fully adhere to the given algorithms. To address this limitation, this paper presents corrupted learning dynamics that adaptively find an equilibrium at a rate dependent on the degree of deviation by each player from the given algorithm's output. First, in two-player zero-sum games, we provide learning dynamics where the external regret of the x-player (and similarly for the y-player) in the corrupted regime is roughly bounded by $O(\log (m_\mathrm{x} m_\mathrm{y}) + \sqrt{C_\mathrm{y}} + C_\mathrm{x})$, which implies a convergence rate of $\tilde{O}((\sqrt{C_\mathrm{y}} + C_\mathrm{x})/T)$ to a Nash equilibrium. Here, $m_\mathrm{x}$ and $m_\mathrm{y}$ are the number of actions of the x- and y-players, respectively, and $C_\mathrm{x}$ and $C_\mathrm{y}$ are the cumulative deviations of the x- and y-players from their given algorithms. Furthermore, we extend our approach to multi-player general-sum games, showing that the swap regret of player $i$ in the corrupted regime is bounded by $O(\log T + \sqrt{\sum_j C_j \log T} + C_i)$, where $C_i$ is the cumulative deviations of player $i$ from the given algorithm. This implies a convergence rate of $O((\log T + \sqrt{\sum_j C_j \log T} + C_i)/T)$ to a correlated equilibrium. Our learning dynamics are agnostic to the corruption levels and are based on OFTRL with new adaptive learning rates.

Learning not to Regret

RM-FSP: Regret Minimization Optimizes Neural Fictitious Self-Play

Is Learning in Games Good for the Learners?

Regret Pruning for Learning Equilibria in Simulation-Based Games

Meta-Learning in Games

Evolutionary Dynamics and $Φ$-Regret Minimization in Games

No-Regret Learning in Time-Varying Zero-Sum Games

Corrupted Learning Dynamics in Games

Optimize Neural Fictitious Self-Play in Regret Minimization Thinking

Last-Iterate Convergence Properties of Regret-Matching Algorithms in Games

On the Convergence of No-Regret Learning Dynamics in Time-Varying Games

No Regrets: Investigating and Improving Regret Approximations for Curriculum Discovery

ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regret

Geometrical Regret Matching

Iterated Regret Minimization: A More Realistic Solution Concept

Near-Optimal $Φ$-Regret Learning in Extensive-Form Games

Equilibrium Finding in Normal-Form Games Via Greedy Regret Minimization

Penalty-Regulated Dynamics and Robust Learning Procedures in Games

Integrating Dynamic Weighted Approach with Fictitious Play and Pure Counterfactual Regret Minimization for Equilibrium Finding

Impact of Decentralized Learning on Player Utilities in Stackelberg Games

No-Regret Learning in Network Stochastic Zero-Sum Games