Abstract:Recent advances at the intersection of dense large graph limits and mean field games have begun to enable the scalable analysis of a broad class of dynamical sequential games with large numbers of agents. So far, results have been largely limited to graphon mean field systems with continuous-time diffusive or jump dynamics, typically without control and with little focus on computational methods. We propose a novel discrete-time formulation for graphon mean field games as the limit of non-linear dense graph Markov games with weak interaction. On the theoretical side, we give extensive and rigorous existence and approximation properties of the graphon mean field solution in sufficiently large systems. On the practical side, we provide general learning schemes for graphon mean field equilibria by either introducing agent equivalence classes or reformulating the graphon mean field system as a classical mean field system. By repeatedly finding a regularized optimal control solution and its generated mean field, we successfully obtain plausible approximate Nash equilibria in otherwise infeasible large dense graph games with many agents. Empirically, we are able to demonstrate on a number of examples that the finite-agent behavior comes increasingly close to the mean field behavior for our computed equilibria as the graph or system size grows, verifying our theory. More generally, we successfully apply policy gradient reinforcement learning in conjunction with sequential Monte Carlo methods.

Mean-field theory of learning: from dynamics to statics

Mean-Field Learning: a Survey

Dynamical and stationary properties of on-line learning from finite training sets

Recent Applications of Dynamical Mean-Field Methods

Learning in mean field games: The fictitious play

Mean-field Analysis of Generalization Errors

Theoretical bound of the efficiency of learning with coarse-graining

Rigorous dynamical mean field theory for stochastic gradient descent methods

Introduction to dynamical mean-field theory of randomly connected neural networks with bidirectionally correlated couplings

Mean Field Games and Nonlinear Markov Processes

Machine learning for many-body physics: efficient solution of dynamical mean-field theory

Learning Graphon Mean Field Games and Approximate Nash Equilibria

Mean Field Theory of Dynamical Systems Driven by External Signals

Dynamic mean field programming

Advanced Mean Field Theory of Restricted Boltzmann Machine

Computational Mean-field information dynamics associated with Reaction diffusion equations

Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification

Learning about learning by many-body systems

Learning Correlated Equilibria in Mean-Field Games

Stochastic weight matrix dynamics during learning and Dyson Brownian motion

On kernel-based statistical learning in the mean field limit