Abstract:Stackelberg games originate where there are market leaders and followers, and the actions of leaders influence the behavior of the followers. Mathematical modelling of such games results in what's called a Bilevel Optimization problem. There is an entire area of research dedicated to analyzing and solving Bilevel Optimization problems which are often complex, and finding solutions for such problems is known to be NP-Hard. A generalization of Stackelberg games is a Multilevel Stackelberg game where we may have nested leaders and followers, such that a follower is, in turn, a leader for all lower-level players. These problems are much more difficult to solve, and existing solution approaches typically require extensive cooperation between the players (which generally can't be assumed) or make restrictive assumptions about the structure of the problem. In this paper, we present a stochastic algorithm to approximate the local equilibrium solutions for these Multilevel games. We then construct a few examples of such Multilevel problems, including: a) a nested toll-setting problem; and b) an adversarial initial condition determination problem for Robust Trajectory Optimization. We test our algorithm on our constructed problems as well as some trilevel problems from the literature, and show that it is able to approximate the optimum solutions for these problems within a reasonable error margin. We also provide an asymptotic proof for the convergence of the algorithm and empirically analyze its accuracy and convergence speed for different parameters. Lastly, we compare it with existing solution strategies from the literature and demonstrate that it outperforms them.

A Machine Learning Method for Stackelberg Mean Field Games

A machine learning framework for solving high-dimensional mean field game and mean field control problems

Direct Approach of Linear-Quadratic Stackelberg Mean Field Games of Backward-Forward Stochastic Systems

Learning in Mean Field Games: A Survey

Deep Learning for Population-Dependent Controls in Mean Field Control Problems with Common Noise

Learning in Herding Mean Field Games: Single-Loop Algorithm with Finite-Time Convergence Analysis

Monte Carlo Optimization for Solving Multilevel Stackelberg Games

Deep Learning for Mean Field Games with non-separable Hamiltonians

Master equation of discrete-time Stackelberg mean field games with multiple leaders

From Nash Equilibrium to Social Optimum and vice versa: a Mean Field Perspective

Learning Discrete-Time Major-Minor Mean Field Games

A Single Online Agent Can Efficiently Learn Mean Field Games

A General Framework for Learning Mean-Field Games

Linear-Quadratic Mean Field Stackelberg Stochastic Differential Game with Partial Information and Common Noise

Solving Strongly Convex and Smooth Stackelberg Games Without Modeling the Follower

Mean Field Games in a Stackelberg Problem with an Informed Major Player

Model-free Reinforcement Learning for Non-stationary Mean Field Games

Alternating the Population and Control Neural Networks to Solve High-Dimensional Stochastic Mean-Field Games

Stackelberg Mean Field Games: convergence and existence results to the problem of Principal with multiple Agents in competition

Machine Learning Methods for Large Population Games with Applications in Operations Research