Abstract:In this article, a data-efficient off-policy reinforcement learning (RL) approach is proposed for distributed output tracking control of heterogeneous multiagent systems (HMASs) using approximate dynamic programming (ADP). Different from existing results that the kinematic model of the exosystem is addressable to partial or all agents, the dynamics of the exosystem are assumed to be completely unknown for all agents in this article. To solve this difficulty, an identifiable algorithm using the experience-replay method is designed for each agent to identify the system matrices of the novel reference model instead of the original exosystem. Then, an output-based distributed adaptive output observer is proposed to provide the estimations of the leader, and the proposed observer not only has a low dimension and less data transmission among agents but also is implemented in a fully distributed way. Besides, a data-efficient RL algorithm is given to design the optimal controller offline along with the system trajectories without solving output regulator equations. An ADP approach is developed to iteratively solve game algebraic Riccati equations (GAREs) using online information of state and input in an online way, which relaxes the requirement of knowing prior knowledge of agents' system matrices in an offline way. Finally, a numerical example is provided to verify the effectiveness of theoretical analysis.

Robust Tracking Control and Output Regulation

A Learning-Based Optimal Tracking Controller for Continuous Linear Systems with Unknown Dynamics: Theory and Case Study

Robust Output Regulation and Reinforcement Learning-Based Output Tracking Design for Unknown Linear Discrete-Time Systems

Data-Efficient Off-Policy Learning for Distributed Optimal Tracking Control of HMAS with Unidentified Exosystem Dynamics.

Adaptive Optimal Output-Feedback Consensus Tracking Control of Nonlinear Multiagent Systems Using Two-Player Stackelberg Game

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Robust Integral Of Neural Network And Sign Of Tracking Error Control Of Uncertain Nonlinear Affine Systems Using State And Output Feedback

Dynamic event-triggered robust optimal tracking control for multi-player nonzero-sum games with mismatched uncertainties and asymmetric constrained inputs

Cooperative Path Following Control in Autonomous Vehicles Graphical Games: A Data-Based Off-Policy Learning Approach

Online Reinforcement Learning-based Neural Network Controller Design for Affine Nonlinear Discrete-time Systems.

The Adaptive Optimal Output Feedback Tracking Control of Unknown Discrete-Time Linear Systems Using a Multistep Q-Learning Approach

Output‐feedback Q‐learning for discrete‐time linear <i>H</i><sup>∞</sup> tracking control: A Stackelberg game approach

Output-feedback Q-learning for discrete-time linear H-infinity tracking control: A Stackelberg game approach

Robust Reinforcement Learning for Risk-Sensitive Linear Quadratic Gaussian Control

Reinforcement learning for optimal tracking of large-scale systems with multitime scales

Optimal Tracking Control of Nonlinear Multiagent Systems Using Internal Reinforce Q-Learning

Model-Free Cooperative Optimal Output Regulation for Linear Discrete-Time Multi-Agent Systems Using Reinforcement Learning

Cooperative output feedback tracking control of stochastic nonlinear heterogeneous multi‐agent systems

Event‐triggered optimal tracking control of multiplayer unknown nonlinear systems via adaptive critic designs

Specified convergence rate guaranteed output tracking of discrete-time systems via reinforcement learning

Optimal Robust Online Tracking Control for Space Manipulator in Task Space Using Off-Policy Reinforcement Learning