Abstract:Consider $N$ players each with a $d$-dimensional action set. Each of the players' utility functions includes their reward function and a linear term for each dimension, with coefficients that are controlled by the manager. We assume that the game is strongly monotone, so if each player runs gradient descent, the dynamics converge to a unique Nash equilibrium (NE). The NE is typically inefficient in terms of global performance. The resulting global performance of the system can be improved by imposing $K$-dimensional linear constraints on the NE. We therefore want the manager to pick the controlled coefficients that impose the desired constraint on the NE. However, this requires knowing the players' reward functions and their action sets. Obtaining this game structure information is infeasible in a large-scale network and violates the users' privacy. To overcome this, we propose a simple algorithm that learns to shift the NE of the game to meet the linear constraints by adjusting the controlled coefficients online. Our algorithm only requires the linear constraints violation as feedback and does not need to know the reward functions or the action sets. We prove that our algorithm, which is based on two time-scale stochastic approximation, guarantees convergence with probability 1 to the set of NE that meet target linear constraints. We then provide a mean square convergence rate of $O(t^{-1/4})$ for our algorithm. This is the first such bound for two time-scale stochastic approximation where the slower time-scale is a fixed point iteration with a non-expansive mapping. We demonstrate how our scheme can be applied to optimizing a global quadratic cost at NE and load balancing in resource allocation games. We provide simulations of our algorithm for these scenarios.

Approximate N-Player Nonzero-Sum Game Solution for an Uncertain Continuous Nonlinear System

Nonzero-Sum Games Using Actor-Critic Neural Networks: A Dynamic Event-Triggered Adaptive Dynamic Programming

Model-Free Adaptive Optimal Control for Unknown Nonlinear Multiplayer Nonzero-Sum Game

An efficient model‐free adaptive optimal control of continuous‐time nonlinear non‐zero‐sum games based on integral reinforcement learning with exploration

Event-Triggered ADP for Nonzero-Sum Games of Unknown Nonlinear Systems

Discrete-Time Nonzero-Sum Games for Multiplayer Using Policy-Iteration-Based Adaptive Dynamic Programming Algorithms

Adaptive Dynamic Programming for a Nonlinear Two‐Player Non‐Zero‐Sum Differential Game With State and Input Constraints

Differential-game for resource aware approximate optimal control of large-scale nonlinear systems with multiple players

Monte Carlo Neural Fictitious Self-Play: Achieve Approximate Nash equilibrium of Imperfect-Information Games.

Approximate-optimal control algorithm for constrained zero-sum differential games through event-triggering mechanism

Online reinforcement learning control of unknown nonaffine nonlinear discrete time systems

Online Adaptive Optimal Control for Continuous-Time Nonlinear Systems with Completely Unknown Dynamics.

Algorithm for Computing Approximate Nash Equilibrium in Continuous Games with Application to Continuous Blotto

Learning to Control Unknown Strongly Monotone Games

Dynamic event-triggered robust optimal tracking control for multi-player nonzero-sum games with mismatched uncertainties and asymmetric constrained inputs

Zero-Sum Game-Based Decentralized Optimal Control for Saturated Nonlinear Interconnected Systems via a Data and Event Driven Approach

Sliding-mode surface-based adaptive optimal nonzero-sum games for saturated nonlinear multi-player systems with identifier-critic networks

Neural-network-based safe learning control for non-zero-sum differential games of nonlinear systems with asymmetric input constraints

Multiplayer Stackelberg-Nash Game for Nonlinear System via Value Iteration-Based Integral Reinforcement Learning

Min–max adaptive dynamic programming for zero-sum differential games

Inverse linear-quadratic nonzero-sum differential games