Abstract:Consider $N$ players each with a $d$-dimensional action set. Each of the players' utility functions includes their reward function and a linear term for each dimension, with coefficients that are controlled by the manager. We assume that the game is strongly monotone, so if each player runs gradient descent, the dynamics converge to a unique Nash equilibrium (NE). The NE is typically inefficient in terms of global performance. The resulting global performance of the system can be improved by imposing $K$-dimensional linear constraints on the NE. We therefore want the manager to pick the controlled coefficients that impose the desired constraint on the NE. However, this requires knowing the players' reward functions and their action sets. Obtaining this game structure information is infeasible in a large-scale network and violates the users' privacy. To overcome this, we propose a simple algorithm that learns to shift the NE of the game to meet the linear constraints by adjusting the controlled coefficients online. Our algorithm only requires the linear constraints violation as feedback and does not need to know the reward functions or the action sets. We prove that our algorithm, which is based on two time-scale stochastic approximation, guarantees convergence with probability 1 to the set of NE that meet target linear constraints. We then provide a mean square convergence rate of $O(t^{-1/4})$ for our algorithm. This is the first such bound for two time-scale stochastic approximation where the slower time-scale is a fixed point iteration with a non-expansive mapping. We demonstrate how our scheme can be applied to optimizing a global quadratic cost at NE and load balancing in resource allocation games. We provide simulations of our algorithm for these scenarios.

A Necessary and Sufficient Condition Beyond Monotonicity for Convergence of the Gradient Play in Continuous Games

Gradient play in stochastic games: stationary points, convergence, and sample complexity

Almost Sure Convergence to Approximate Nash Equilibrium in Zero-Sum Extensive-Form Games with Noisy Feedback

On Gradient-Based Learning in Continuous Games

Convergence of Policy Gradient Methods for Nash Equilibria in General-sum Stochastic Games

On the Convergence of Distributed Projected Gradient Play with Heterogeneous Learning Rates in Monotone Games.

Exponential Convergence of Gradient Methods in Concave Network Zero-sum Games

Linear Last-Iterate Convergence for Continuous Games with Coupled Inequality Constraints

Geometric Convergence of Gradient Play Algorithms for Distributed Nash Equilibrium Seeking

Convergence of Learning Dynamics in Stackelberg Games

Linear Last-Iterate Convergence for Continuous Games with Coupled Inequality Constraints

Passivity-based Gradient-Play Dynamics for Distributed Generalized Nash Equilibrium Seeking

Convergence of $\text{log}(1/ε)$ for Gradient-Based Algorithms in Zero-Sum Games without the Condition Number: A Smoothed Analysis

Learning in games with continuous action sets and unknown payoff functions

On the Rate of Convergence of Continuous-Time Game Dynamics in N-Player Potential Games

CONTINUOUS-TIME CONVERGENCE RATES IN POTENTIAL AND MONOTONE GAMES

Decentralized Policy Gradient for Nash Equilibria Learning of General-sum Stochastic Games

Approaching the Global Nash Equilibrium of Non-convex Multi-player Games

Learning to Control Unknown Strongly Monotone Games

On Convergence Rates of Robust Adaptive Game Theoretic Learning Algorithms

Global Nash Equilibrium in Non-convex Multi-player Game: Theory and Algorithms