Learning Equilibrium with Estimated Payoffs in Population Games

Shinkyu Park

2024-09-16

Abstract:We study a multi-agent decision problem in population games, where agents select from multiple available strategies and continually revise their selections based on the payoffs associated with these strategies. Unlike conventional population game formulations, we consider a scenario where agents must estimate the payoffs through local measurements and communication with their neighbors. By employing task allocation games -- dynamic extensions of conventional population games -- we examine how errors in payoff estimation by individual agents affect the convergence of the strategy revision process. Our main contribution is an analysis of how estimation errors impact the convergence of the agents' strategy profile to equilibrium. Based on the analytical results, we propose a design for a time-varying strategy revision rate to guarantee convergence. Simulation studies illustrate how the proposed method for updating the revision rate facilitates convergence to equilibrium.

Multiagent Systems,Systems and Control

What problem does this paper attempt to address?

This paper attempts to solve the problem of how estimation errors affect the convergence of the strategy revision process to the equilibrium state in population games when decision - making entities (i.e., agents) need to estimate the payoff vector through local measurements and communication with neighbors. Specifically, the researchers focus on how agents select and adjust strategies based on payoff estimates in the framework of the task - assignment game, a dynamically extended population game, and explore the specific impacts of these estimation errors on the convergence of the strategy revision process. The main contribution of the paper lies in analyzing how estimation errors affect the convergence of agent strategy configurations to equilibrium, and based on this, proposes a time - varying strategy revision rate design method to ensure convergence. Through simulation studies, the author shows how the proposed revision rate update method promotes the convergence of the strategy revision process to the equilibrium state.

Learning Equilibrium with Estimated Payoffs in Population Games

Learning with Delayed Payoffs in Population Games using Kullback-Leibler Divergence Regularization

Payoff Mechanism Design for Coordination in Multi-Agent Task Allocation Games

Game Dynamics and Equilibrium Computation in the Population Protocol Model

Learning, evolution and population dynamics

Social Optimum Equilibrium Selection for Distributed Multi-Agent Optimization

A Risk-Averse Equilibrium for Multi-Agent Systems

Tuning Cooperative Behavior in Games with Nonlinear Opinion Dynamics

Chaos persists in large-scale multi-agent learning despite adaptive learning rates

Score-Based Equilibrium Learning in Multi-Player Finite Games with Imperfect Information

Penalty-Regulated Dynamics and Robust Learning Procedures in Games

Adaptive Incentive Design with Learning Agents

Learning Equilibria of Simulation-Based Games

Epidemic Population Games And Perturbed Best Response Dynamics

Learning enables adaptation in cooperation for multi-player stochastic games

Incentive Designs for Learning Agents to Stabilize Coupled Exogenous Systems

Reinforcement Learning for Non-stationary Discrete-Time Linear–Quadratic Mean-Field Games in Multiple Populations

Evolutionary matrix-game dynamics under imitation in heterogeneous populations

Improving Quantal Cognitive Hierarchy Model Through Iterative Population Learning

Evolutionary Dynamics of Population Games With an Aspiration-Based Learning Rule

A study of factors in the formation of population game cooperation based on mixed learning rules