Abstract:This paper integrates game theory, optimal control theory and reinforcement learning to deal with the discrete-time (DT) multi-player non-zero-sum game issue. As is known, the solutions to non-zero-sum game problems are the outcomes of coupled Riccati equations or coupled Hamilton–Jacobi ones, which are generally difficult to solve analytically and require the knowledge of accurate system mathematical models. However, for most practical industrial systems, the system dynamics cannot be obtained accurately or even unavailable, and the conventional model-based methods will be invalid. To overcome this deficiency, we develop data-based adaptive dynamic programming (ADP) algorithms for completely unknown multi-player systems. Firstly, the Nash equilibrium and stationarity conditions are used to formulate the DT multi-player non-zero-sum game, and then policy iteration algorithm is applied to approximate optimal solutions successively. Secondly, a novel online ADP algorithm combined with a neural-network-based identification scheme is designed and only requires the system data instead of the real system models. Subsequently, a data-driven action-dependent heuristic dynamic programming approach is presented and circumvents the estimation errors caused by the identification learning procedure. Finally, two simulation examples are provided to illustrate the feasibility of our schemes.

A Summary on Some Typical Adaptive Dynamic Programming Schemes

Adaptive dynamic programming: an introduction

On Distributed Implementation of Switch-Based Adaptive Dynamic Programming

An Overview of Research on Adaptive Dynamic Programming

Iterative ADP Learning Algorithms for Discrete-Time Multi-Player Games.

Online Sparse Kernel Learning-Based Adaptive Dynamic Programming

Research of a Parallel Learning Adaptive Dynamic Programming Based on Genetic Algorithms

A Parallel Framework of Adaptive Dynamic Programming Algorithm with Off-Policy Learning.

Adaptive Dynamic Programming for Control: A Survey and Recent Advances

Data-driven Adaptive Dynamic Programming Schemes for Non-Zero-sum Games of Unknown Discrete-Time Nonlinear Systems

An integrated design for intensified direct heuristic dynamic programming

Hamiltonian-driven Adaptive Dynamic Programming for Nonlinear Discrete-Time Dynamic Systems.

A CUDA-based Parallel Adaptive Dynamic Programming Algorithm

Adaptive Dynamic Programming Based on Parallel Control Theory for Underwater Vehicles.

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Twin Deterministic Policy Gradient Adaptive Dynamic Programming for Optimal Control of Affine Nonlinear Discrete-time Systems

On Iterative Adaptive Dynamic Programming

Neural-network-based Learning Algorithms for Cooperative Games of Discrete-Time Multi-Player Systems with Control Constraints Via Adaptive Dynamic Programming

Finite-approximation-error-based Discrete-Time Iterative Adaptive Dynamic Programming.

An Adaptive Dynamic Programming Control Scheme Using Tunable Radial Basis Function

Heuristic Dynamic Programming Strategy with Eligibility Traces