Abstract:In this paper, we examine the long-run behavior of regularized, no-regret learning in finite games. A well-known result in the field states that the empirical frequencies of no-regret play converge to the game's set of coarse correlated equilibria; however, our understanding of how the players' actual strategies evolve over time is much more limited - and, in many cases, non-existent. This issue is exacerbated further by a series of recent results showing that only strict Nash equilibria are stable and attracting under regularized learning, thus making the relation between learning and pointwise solution concepts particularly elusive. In lieu of this, we take a more general approach and instead seek to characterize the \emph{setwise} rationality properties of the players' day-to-day play. To that end, we focus on one of the most stringent criteria of setwise strategic stability, namely that any unilateral deviation from the set in question incurs a cost to the deviator - a property known as closedness under better replies (club). In so doing, we obtain a far-reaching equivalence between strategic and dynamic stability: a product of pure strategies is closed under better replies if and only if its span is stable and attracting under regularized learning. In addition, we estimate the rate of convergence to such sets, and we show that methods based on entropic regularization (like the exponential weights algorithm) converge at a geometric rate, while projection-based methods converge within a finite number of iterations, even with bandit, payoff-based feedback.

On the stability of an adaptive learning dynamics in traffic games

Complex Dynamics and Control of A Dynamic R&D Bertrand Triopoly Game Model with Bounded Rational Rule

Stabilization of Traffic Flow Based on Multi-Anticipative Intelligent Driver Model

Penalty-Regulated Dynamics and Robust Learning Procedures in Games

On the Resilience of Traffic Networks under Non-Equilibrium Learning

Imitation dynamics in a game of traffic

On the robustness of learning in games with stochastically perturbed payoff observations

The equivalence of dynamic and strategic stability under regularized learning in games

Navigation Systems May Deteriorate Stability in Traffic Networks

Local stability of traffic equilibria in an isotropic network

Best-response dynamics, playing sequences, and convergence to equilibrium in random games

On the stability of the logit dynamics in population games

Gradient Dynamics in Linear Quadratic Network Games with Time-Varying Connectivity and Population Fluctuation

Solving N-player dynamic routing games with congestion: a mean field approach

Linearly Solvable Mean-Field Road Traffic Games

Stability of the Nash Equilibrium under Gradient Ascent Learning Algorithms in Two-Agent Two-Action Games

Stability of Multi-Agent Learning in Competitive Networks: Delaying the Onset of Chaos

Dynamic games on arbitrary networks with two types of players

On Convergence Rates of Robust Adaptive Game Theoretic Learning Algorithms

A stability property in mean field type differential games

On the Stability of Learning in Network Games with Many Players