Abstract:In this paper, an off-policy game Q-learning algorithm is proposed for solving linear discrete-time non-zero sum multi-player game problems. Unlike the existing Q-learning methods for solving the Riccati equation by on-policy learning approaches for multi-player games, an off-policy game Q-learning method is developed for achieving the Nash equilibrium of multiple players. To this end, first, a non-zero sum game problem is formulated, and the value function and the Q-function defined according to each-player individual performance index are rigorously proved to be linear quadratic forms. Then, based on the dynamic programming and Q-learning methods, an off-policy game Q-learning algorithm is developed to find the control policies for multi-player games, such that the Nash equilibrium is reached under the learned control policies. The merit of this paper lies in that the proposed algorithm does not require the system model parameters to be known a priori and fully utilizes measurable data to learn the Nash equilibrium solution. Moreover, there is no bias of Nash equilibrium solution when implementing the proposed off-policy game Q-learning algorithm even though probing noises are added to control policies for maintaining the persistent excitation condition. While bias of the Nash equilibrium solution could be produced if on-policy game Q-learning is employed. This is another contribution of this paper

Off-Policy Integral Reinforcement Learning Method to Solve Nonlinear Continuous-Time Multiplayer Nonzero-Sum Games.

Integral Reinforcement Learning Off-Policy Method for Solving Nonlinear Multi-Player Nonzero-Sum Games with Saturated Actuator.

Off-policy Integral Reinforcement Learning Algorithm in Dealing with Nonzero Sum Game for Nonlinear Distributed Parameter Systems.

IRL Method for Time-Continuous Two-Player Nonzero Sum Game of Unknown System with Constrained-Input

Mix-zero-sum Differential Games for Linear Systems with Unknown Dynamics Based on Off-Policy IRL.

Nearly Optimal Control for Mixed Zero-Sum Game Based on Off-Policy Integral Reinforcement Learning

Off-policy Synchronous Iteration IRL Method for Multi-Player Zero-Sum Games with Input Constraints

Discrete-Time Non-Zero-Sum Games With Completely Unknown Dynamics

Integral Reinforcement Learning-Based Optimal Control for Nonzero-Sum Games of Multi-Player Input-Constrained Nonlinear Systems

Event-based Integral Reinforcement Learning Algorithm for Non-zero-sum Games of Partially Unknown Nonlinear Systems

Integral Reinforcement Learning for Linear Continuous-Time Zero-Sum Games With Completely Unknown Dynamics

Integral Policy Iteration for Zero-Sum Games with Completely Unknown Nonlinear Dynamics

Integral Reinforcement Learning-Based Online Adaptive Event-Triggered Control for Non-Zero-sum Games of Partially Unknown Nonlinear Systems.

Data-Driven Integral Reinforcement Learning for Continuous-Time Non-Zero-Sum Games

Optimal Tracking Control for Multi-player Non-Zero-Sum Games of Continuous-Time Linear Systems with Unknown Dynamics.

Data-Driven Nonzero-Sum Game for Discrete-Time Systems Using Off-Policy Reinforcement Learning

Online reinforcement learning multiplayer non-zero sum games of continuous-time Markov jump linear systems

Policy Iteration Based Q-learning for Linear Nonzero-Sum Quadratic Differential Games.

Discrete-Time Multi-Player Games Based on Off-Policy Q-Learning

Non‐zero‐sum games of discrete‐time Markov jump systems with unknown dynamics: An off‐policy reinforcement learning method

Off-policy Based Adaptive Dynamic Programming Method for Nonzero-Sum Games on Discrete-Time System