Abstract:This paper focuses on the optimal output synchronization control problem of heterogeneous multiagent systems (HMASs) subject to nonidentical communication delays by a reinforcement learning method. Compared with existing studies assuming that the precise model of the leader is globally or distributively accessible to all or some of the followers, the leader's precise dynamical model is entirely inaccessible to all the followers in this paper. A data-based learning algorithm is first proposed to reconstruct the leader's unknown system matrix online. A distributed predictor subject to communication delays is further devised to estimate the leader's state, where interaction delays are allowed to be nonidentical. Then, a learning-based local controller, together with a discounted performance function, is projected to reach the optimal output synchronization. Bellman equations and game algebraic Riccati equations are constructed to learn the optimal solution by developing a model-based reinforcement learning (RL) algorithm online without solving regulator equations, which is followed by a model-free off-policy RL algorithm to relax the requirement of all agents' dynamics faced by the model-based RL algorithm. The optimal tracking control of HMASs subject to unknown leader dynamics and communication delays is shown to be solvable under the proposed RL algorithms. Finally, the effectiveness of theoretical analysis is verified by numerical simulations.

Learning-Based Distributed Robust Formation Control Framework for Heterogeneous Multi-Agent Systems under Disturbances.

Distributed Leader-follower Formation Control of Nonholonomic Mobile Robots

Distributed output formation tracking control of heterogeneous multi-agent systems using reinforcement learning

Optimal robust formation control for heterogeneous multi‐agent systems based on reinforcement learning

Resilient Output Formation-Containment Tracking of Heterogeneous Multi-Agent Systems: A Learning-Based Framework using Dynamic Data

Formation Control of Heterogeneous Discrete-Time Nonlinear Multi-Agent Systems with Uncertainties

Group Formation Tracking of Heterogeneous Multi-Agent Systems Using Reinforcement Learning

Distributed robust adaptive formation control of multi-agent systems with heterogeneous uncertainties and directed graphs

Reinforcement learning intermittent optimal formation control for multi-agent systems with disturbances

Heterogeneous formation control of multiple rotorcrafts with unknown dynamics by reinforcement learning

Reinforcement Learning-Based Unknown Reference Tracking Control of HMASs with Nonidentical Communication Delays

Robust distributed formation control with prescribed performance for nonlinear multi‐agent systems subjected to compound disturbances

Robust Formation Tracking Control for Noncooperative Heterogeneous Multiagent Systems

Fully Distributed Robust Adaptive Formation Control for Linear Multi-agent System with Uncertainties and Communication Delays⋆

Adaptive Practical Optimal Time-Varying Formation Tracking Control for Disturbed High-Order Multi-Agent Systems

Heterogeneous formation control of multiple UAVs with limited-input leader via reinforcement learning

Adaptive Time-Varying Formation Maneuvering Control for Multi-Robot Systems in Complex Obstacle-Rich Environments

Relative Distributed Formation and Obstacle Avoidance with Multi-agent Reinforcement Learning

Distributed Multi-layer Time-varying Output Formation Tracking Control for Heterogeneous Linear Multiagent Systems

Model‐free adaptive formation control for unknown multiinput‐multioutput nonlinear heterogeneous discrete‐time multiagent systems with bounded disturbance

Robust Formation Tracking for Uncertain Multi-Agent Systems with Unknown Leader Input