Abstract:Efficient allocation is important in nature and human society where individuals often compete for finite resources. The Minority Game is perhaps the simplest model that provides deep insights into how human coordinate to maximize the resource utilization. However, this model assumes the static strategies that are provided a priori, failing to capture their adaptive nature. Here, we turn to the paradigm of reinforcement learning, where individuals' strategies are evolving by evaluating both the past experience and rewards in the future. Specifically, we adopt the Q-learning algorithm, each player is endowed with a Q-table that guides their decision-making. We reveal that the population is able to reach the optimal allocation when individuals appreciate both the past experience and rewards in the future, and they are able to balance the exploitation of their Q-tables and the exploration by randomly acting. The optimal allocation is ruined when individuals tend to use either exploitation-only or exploration-only, where only partial coordination and even anti-coordination are observed. Mechanism analysis reveals that a moderate level of exploration can escape local minimums of metastable periodic states, and reaches the optimal coordination as the global minimum. Interestingly, the optimal coordination is underlined by a symmetry-breaking of action preferences, where nearly half of the population choose one side while the other half prefer the other side. The emergence of optimal coordination is robust to the population size and other game parameters. Our work therefore provides a natural solution to the Minority Game and sheds insights into the resource allocation problem in general. Besides, our work demonstrates the potential of the proposed reinforcement learning paradigm in deciphering many puzzles in the socio-economic context.

Learning, competition and cooperation in simple games

Social and individual learning in the Minority Game

The Minority Game with interactions

The Local Minority Game

Crowd-Anticrowd Theory of Multi-Agent Minority Games

The minority game: An economics perspective

Decision Making, Strategy dynamics, and Crowd Formation in Agent-based models of Competing Populations

Coordination, intermittency and trends in generalized Minority Games

Evolutionary minority game with heterogeneous strategy distribution

Self-organization, resources and strategies in a minority game

Cooperation and competition between pair and multi-player social games in spatial populations

Evolution in Minority Games II. Games with Variable Strategy Spaces

Learning Fair Cooperation in Mixed-Motive Games with Indirect Reciprocity

A study of factors in the formation of population game cooperation based on mixed learning rules

Self organization in a minority game: the role of memory and a probabilistic approach

Cooperation in the Mixed Population Minority Game with Imitation

Global optimization of minority game by intelligent agents

Effects of Imitation in a Competing and Evolving Population

Self-segregation and enhanced cooperation in an evolving population through local information transmission

Optimal coordination in Minority Game: A solution from reinforcement learning

Cooperation and self-regulation in a model of agents playing different games