Abstract:In this article, we study a multiplayer Stackelberg-Nash game (SNG) pertaining to a nonlinear dynamical system, including one leader and multiple followers. At the higher level, the leader makes its decision preferentially with consideration of the reaction functions of all followers, while, at the lower level, each of the followers reacts optimally to the leader's strategy simultaneously by playing a Nash game. First, the optimal strategies for the leader and the followers are derived from down to the top, and these strategies are further shown to constitute the Stackelberg-Nash equilibrium points. Subsequently, to overcome the difficulty in calculating the equilibrium points analytically, we develop a novel two-level value iteration-based integral reinforcement learning (VI-IRL) algorithm that relies only upon partial information of system dynamics. We establish that the proposed method converges asymptotically to the equilibrium strategies under the weak coupling conditions. Moreover, we introduce effective termination criteria to guarantee the admissibility of the policy (strategy) profile obtained from a finite number of iterations of the proposed algorithm. In the implementation of our scheme, we employ neural networks (NNs) to approximate the value functions and invoke the least-squares methods to update the involved weights. Finally, the effectiveness of the developed algorithm is verified by two simulation examples.

Policy Iteration Adaptive Dynamic Programming for Optimal Control of Multi-Player Stackelberg-Nash Games

Event-Triggered Robust Adaptive Dynamic Programming for Multiplayer Stackelberg-Nash Games of Uncertain Nonlinear Systems

Multi-Player Robust Control of Stackelberg Games via Adaptive Dynamic Programming

Neural-network-based Learning Algorithms for Cooperative Games of Discrete-Time Multi-Player Systems with Control Constraints Via Adaptive Dynamic Programming

Robust ADP-based Control for Uncertain Nonlinear Stackelberg Games

Adaptive Dynamic Programming for a Class of Two-player Stackelberg Differential Games.

Observer-Based Adaptive Dynamic Programming Control for Stackelberg Differential Games of Input Constrained Two-Player Nonlinear Systems*

Iterative ADP Learning Algorithms for Discrete-Time Multi-Player Games.

Adaptive Optimal Control Via Continuous-Time $Q$-Learning for Stackelberg–Nash Games of Uncertain Nonlinear Systems

Online Finite-Horizon Optimal Learning Algorithm for Nonzero-Sum Games with Partially Unknown Dynamics and Constrained Inputs

Multiplayer Stackelberg-Nash Game for Nonlinear System via Value Iteration-Based Integral Reinforcement Learning

Discrete-Time Nonzero-Sum Games for Multiplayer Using Policy-Iteration-Based Adaptive Dynamic Programming Algorithms

Sliding-mode surface-based approximate optimal control for nonlinear multiplayer Stackelberg-Nash games via adaptive dynamic programming

Adaptive Dynamic Programming for Solving Non-Zero-Sum Differential Games.

Data-driven Adaptive Dynamic Programming Schemes for Non-Zero-sum Games of Unknown Discrete-Time Nonlinear Systems

Policy-Iteration-Based Learning for Nonlinear Player Game Systems with Constrained Inputs.

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Online Dual-Network-Based Adaptive Dynamic Programming for Solving Partially Unknown Multi-Player Non-Zero-Sum Games with Control Constraints

Optimal consensus control for multi-agent systems: Multi-step policy gradient adaptive dynamic programming method

Policy Gradient Adaptive Dynamic Programming for Nonlinear Discrete-Time Zero-Sum Games with Unknown Dynamics

Local Policy Iteration Adaptive Dynamic Programming for Discrete-Time Nonlinear Systems