Abstract:This paper focuses on the optimal output synchronization control problem of heterogeneous multiagent systems (HMASs) subject to nonidentical communication delays by a reinforcement learning method. Compared with existing studies assuming that the precise model of the leader is globally or distributively accessible to all or some of the followers, the leader's precise dynamical model is entirely inaccessible to all the followers in this paper. A data-based learning algorithm is first proposed to reconstruct the leader's unknown system matrix online. A distributed predictor subject to communication delays is further devised to estimate the leader's state, where interaction delays are allowed to be nonidentical. Then, a learning-based local controller, together with a discounted performance function, is projected to reach the optimal output synchronization. Bellman equations and game algebraic Riccati equations are constructed to learn the optimal solution by developing a model-based reinforcement learning (RL) algorithm online without solving regulator equations, which is followed by a model-free off-policy RL algorithm to relax the requirement of all agents' dynamics faced by the model-based RL algorithm. The optimal tracking control of HMASs subject to unknown leader dynamics and communication delays is shown to be solvable under the proposed RL algorithms. Finally, the effectiveness of theoretical analysis is verified by numerical simulations.

Leader–Follower Coordinated Tracking of Multiple Heterogeneous Lagrange Systems Using Continuous Control

Zero-Error Coordinated Tracking Of Multiple Lagrange Systems Using Continuous Control

Distributed Leader-follower Formation Control of Nonholonomic Mobile Robots

Coordinated Tracking Of Multi-Agent Systems With A Leader Of Bounded Unknown Input Using Distributed Continuous Controllers

Distributed Adaptive Tracking Control for Unknown Nonlinear Networked Systems

Observer-Based Event-Triggered Tracking Control of Leader-Follower Systems with Time Delay

Coordinated Tracking of Euler-Lagrange Systems over Directed Graphs Via Distributed Continuous Controllers

Leader–follower Swarm Tracking for Networked Lagrange Systems

Distributed Event-Triggered Tracking Control with a Dynamic Leader for Multiple Euler-Lagrange Systems under Directed Networks

Robust Cooperative Tracking for Multiple Non-Identical Second-Order Nonlinear Systems

Distributed Tracking Control for Linear Multiagent Systems with A Leader of Bounded Unknown Input

Distributed Cooperative Tracking for Multiple Second-Order Nonlinear Systems Using Only Relative Position Measurements

Distributed Control of High-Order Nonlinear Input Constrained Multiagent Systems Using a Backstepping-Free Method

Leader-follower Tracking Control with Guaranteed Consensus Performance for Interconnected Systems with Linear Dynamic Uncertain Coupling

Distributed tracking control for linear multi-agent systems with a leader of bounded input using output information

Distributed Adaptive Tracking Control of Hidden Leader-Follower Multi-Agent Systems with Unknown Parameters

Reinforcement Learning-Based Unknown Reference Tracking Control of HMASs with Nonidentical Communication Delays

Adaptive Cooperative Tracking and Parameter Estimation of an Uncertain Leader over General Directed Graphs

Leader–Following Consensus of Multiple Uncertain Euler–Lagrange Systems With Unknown Dynamic Leader

On Distributed Finite-Time Observer Design and Finite-Time Coordinated Tracking of Multiple Double Integrator Systems Via Local Interactions

Distributed Tracking Control Of Multi-Agent Systems With Heterogeneous Uncertainties