Abstract:A key challenge of evolutionary game theory and multi-agent learning is to characterize the limit behavior of game dynamics. Whereas convergence is often a property of learning algorithms in games satisfying a particular reward structure (e.g., zero-sum games), even basic learning models, such as the replicator dynamics, are not guaranteed to converge for general payoffs. Worse yet, chaotic behavior is possible even in rather simple games, such as variants of the Rock-Paper-Scissors game. Although chaotic behavior in learning dynamics can be precluded by the celebrated Poincaré-Bendixson theorem, it is only applicable to low-dimensional settings. Are there other characteristics of a game that can force regularity in the limit sets of learning? We show that behavior consistent with the Poincaré-Bendixson theorem (limit cycles, but no chaotic attractor) can follow purely from the topological structure of the interaction graph, even for high-dimensional settings with an arbitrary number of players and arbitrary payoff matrices. We prove our result for a wide class of follow-the-regularized leader (FoReL) dynamics, which generalize replicator dynamics, for binary games characterized interaction graphs where the payoffs of each player are only affected by one other player (i.e., interaction graphs of indegree one). Since chaos occurs already in games with only two players and three strategies, this class of non-chaotic games may be considered maximal. Moreover, we provide simple conditions under which such behavior translates into efficiency guarantees, implying that FoReL learning achieves time-averaged sum of payoffs at least as good as that of a Nash equilibrium, thereby connecting the topology of the dynamics to social-welfare analysis.

Counterclockwise Dissipativity, Potential Games and Evolutionary Nash Equilibrium Learning

Evolutionary Games on Infinite Strategy Sets: Convergence to Nash Equilibria via Dissipativity

Passivity Tools for Hybrid Learning Rules in Large Populations

Penalty-Regulated Dynamics and Robust Learning Procedures in Games

Evolutionary Game Theory Squared: Evolving Agents in Endogenously Evolving Zero-Sum Games

Passivity and Evolutionary Game Dynamics

Poincaré-Bendixson Limit Sets in Multi-Agent Learning

On Passivity, Reinforcement Learning and Higher-Order Learning in Multi-Agent Finite Games

Effect of reciprocity mechanisms on evolutionary dynamics in feedback-evolving games

On the discrete-time origins of the replicator dynamics: From convergence to instability and chaos

Evolutionary Dynamics of Population Games With an Aspiration-Based Learning Rule

Incentive Designs for Learning Agents to Stabilize Coupled Exogenous Systems

Learning, evolution and population dynamics

A Payoff-Based Learning Approach for Nash Equilibrium Seeking in Continuous Potential Games.

Nested replicator dynamics, nested logit choice, and similarity-based learning

Dynamic Cournot-Nash Equilibrium: the Non-Potential Case

Evolutionary Game Dynamics for Two Interacting Populations in A Co-evolving Environment.

On dynamics and Nash equilibriums of networked games

Evolutionary Game Dynamics of Multi-Agent Cooperation Driven by Self-Learning

Evolutionary network games: equilibria from imitation and best-response dynamics

Evolutionary Dynamics On Stochastic Evolving Networks For Multiple-Strategy Games