Abstract:In this paper we study iterative procedures for stationary equilibria in games with large number of players. Most of learning algorithms for games with continuous action spaces are limited to strict contraction best reply maps in which the Banach-Picard iteration converges with geometrical convergence rate. When the best reply map is not a contraction, Ishikawa-based learning is proposed. The algorithm is shown to behave well for Lipschitz continuous and pseudo-contractive maps. However, the convergence rate is still unsatisfactory. Several acceleration techniques are presented. We explain how cognitive users can improve the convergence rate based only on few number of measurements. The methodology provides nice properties in mean field games where the payoff function depends only on own-action and the mean of the mean-field (first moment mean-field games). A learning framework that exploits the structure of such games, called, mean-field learning, is proposed. The proposed mean-field learning framework is suitable not only for games but also for non-convex global optimization problems. Then, we introduce mean-field learning without feedback and examine the convergence to equilibria in beauty contest games, which have interesting applications in financial markets. Finally, we provide a fully distributed mean-field learning and its speedup versions for satisfactory solution in wireless networks. We illustrate the convergence rate improvement with numerical examples.

Mean field for Markov Decision Processes: from Discrete to Continuous Optimization

Continuous-Time Mean Field Markov Decision Models

Mean Field Markov Decision Processes

Finite Horizon Continuous-Time Markov Decision Processes with Mean and Variance Criteria

Mean Field Games with Mean-Field-Dependent Volatility: A PDE Approach

Mean Field Games Master Equations: From Discrete to Continuous State Space

Extended mean-field control problems with multi-dimensional singular controls

Mean-field control of non exchangeable systems

Introduction to Mean Field Games and Mean-Field-Type Games

A Mean–variance Optimization Problem for Discounted Markov Decision Processes

POMDPs in Continuous Time and Discrete Spaces

Mean-Field Learning: a Survey

Mean-variance optimization of discrete time discounted Markov decision processes.

Discrete Mean Field Games: Existence of Equilibria and Convergence

Closed-loop Equilibria for Mean-Field Games in Randomly Switching Environments with General Discounting Costs

Extended mean-field games with multi-dimensional singular controls and non-linear jump impact

Discrete-time Mean-Field Stochastic Control with Partial Observations

Mean-Field Games with common Poissonian noise: a Maximum Principle approach

Mean-field sparse optimal control