Abstract:Consider $N$ players each with a $d$-dimensional action set. Each of the players' utility functions includes their reward function and a linear term for each dimension, with coefficients that are controlled by the manager. We assume that the game is strongly monotone, so if each player runs gradient descent, the dynamics converge to a unique Nash equilibrium (NE). The NE is typically inefficient in terms of global performance. The resulting global performance of the system can be improved by imposing $K$-dimensional linear constraints on the NE. We therefore want the manager to pick the controlled coefficients that impose the desired constraint on the NE. However, this requires knowing the players' reward functions and their action sets. Obtaining this game structure information is infeasible in a large-scale network and violates the users' privacy. To overcome this, we propose a simple algorithm that learns to shift the NE of the game to meet the linear constraints by adjusting the controlled coefficients online. Our algorithm only requires the linear constraints violation as feedback and does not need to know the reward functions or the action sets. We prove that our algorithm, which is based on two time-scale stochastic approximation, guarantees convergence with probability 1 to the set of NE that meet target linear constraints. We then provide a mean square convergence rate of $O(t^{-1/4})$ for our algorithm. This is the first such bound for two time-scale stochastic approximation where the slower time-scale is a fixed point iteration with a non-expansive mapping. We demonstrate how our scheme can be applied to optimizing a global quadratic cost at NE and load balancing in resource allocation games. We provide simulations of our algorithm for these scenarios.

SLS-BRD: A system-level approach to seeking generalised feedback Nash equilibria

Distributed Generalized Nash Equilibrium Seeking: An Operator-Theoretic Perspective

Nash Equilibria for Linear Quadratic Discrete-time Dynamic Games via Iterative and Data-driven Algorithms

Policies with Sparse Inter-Agent Dependencies in Dynamic Games: A Dynamic Programming Approach

Feedback Nash equilibria for scalar N-player linear quadratic dynamic games

Policy-Iteration-Based Learning for Nonlinear Player Game Systems with Constrained Inputs.

Social Optimum Equilibrium Selection for Distributed Multi-Agent Optimization

The Computation of Approximate Feedback Stackelberg Equilibria in Multiplayer Nonlinear Constrained Dynamic Games

Learning generalized Nash equilibria in monotone games: A hybrid adaptive extremum seeking control approach

The computation of approximate feedback Stackelberg equilibria in multi-player nonlinear constrained dynamic games

Feedback and Open-Loop Nash Equilibria for LQ Infinite-Horizon Discrete-Time Dynamic Games

Learning to Control Unknown Strongly Monotone Games

Learning Stationary Nash Equilibrium Policies in [math]-Player Stochastic Games with Independent Chains

Multiplayer Stackelberg-Nash Game for Nonlinear System via Value Iteration-Based Integral Reinforcement Learning

Penalty-Regulated Dynamics and Robust Learning Procedures in Games

Nash equilibria in scalar discrete-time linear quadratic games

Nash Equilibrium Seeking for General Linear Systems with Disturbance Rejection

A Scalable Game Theoretic Approach for Coordination of Multiple Dynamic Systems

Distributed computation of equilibria in misspecified convex stochastic Nash games

Distributed Nash Equilibrium Seeking for Games in Systems with Unknown Control Directions