Abstract:This paper focuses on the optimal output synchronization control problem of heterogeneous multiagent systems (HMASs) subject to nonidentical communication delays by a reinforcement learning method. Compared with existing studies assuming that the precise model of the leader is globally or distributively accessible to all or some of the followers, the leader's precise dynamical model is entirely inaccessible to all the followers in this paper. A data-based learning algorithm is first proposed to reconstruct the leader's unknown system matrix online. A distributed predictor subject to communication delays is further devised to estimate the leader's state, where interaction delays are allowed to be nonidentical. Then, a learning-based local controller, together with a discounted performance function, is projected to reach the optimal output synchronization. Bellman equations and game algebraic Riccati equations are constructed to learn the optimal solution by developing a model-based reinforcement learning (RL) algorithm online without solving regulator equations, which is followed by a model-free off-policy RL algorithm to relax the requirement of all agents' dynamics faced by the model-based RL algorithm. The optimal tracking control of HMASs subject to unknown leader dynamics and communication delays is shown to be solvable under the proposed RL algorithms. Finally, the effectiveness of theoretical analysis is verified by numerical simulations.

Group Formation Tracking of Heterogeneous Multi-Agent Systems Using Reinforcement Learning

Moving Forward in Formation: A Decentralized Hierarchical Learning Approach to Multi-Agent Moving Together

Distributed output formation tracking control of heterogeneous multi-agent systems using reinforcement learning

Data-Efficient Off-Policy Learning for Distributed Optimal Tracking Control of HMAS with Unidentified Exosystem Dynamics.

Learning-Based Distributed Robust Formation Control Framework for Heterogeneous Multi-Agent Systems under Disturbances.

Reinforcement Learning-Based Unknown Reference Tracking Control of HMASs with Nonidentical Communication Delays

Optimal Tracking Control of Heterogeneous MASs Using Event-Driven Adaptive Observer and Reinforcement Learning

Optimal couple-group tracking control for the heterogeneous multi-agent systems with cooperative-competitive interactions via reinforcement learning method

Formation Tracking of Spatiotemporal Multiagent Systems: A Decentralized Reinforcement Learning Approach

Optimal robust formation control for heterogeneous multi‐agent systems based on reinforcement learning

Zero-Sum Game-Based Controller Design Using Reinforcement Learning for Formation Tracking of Multi-agent Systems

Relative Distributed Formation and Obstacle Avoidance with Multi-agent Reinforcement Learning

Heterogeneous formation control of multiple rotorcrafts with unknown dynamics by reinforcement learning

Time-Varying Formation of Heterogeneous Multiagent Systems via Reinforcement Learning Subject to Switching Topologies

Time-Varying Group Formation Tracking for Multiagent Systems with Competition and Cooperation Via Distributed Nash Equilibrium Seeking

Heterogeneous formation control of multiple UAVs with limited-input leader via reinforcement learning

Resilient Output Formation-Containment Tracking of Heterogeneous Multi-Agent Systems: A Learning-Based Framework using Dynamic Data

Data-Based Output Formation Tracking Control for Heterogeneous Multi-Agent Systems

Practical Optimized Time-Varying Formation Tracking Control for Nonlinear Multi-Agent Systems with Unknown Dynamics and Non-Cooperative Leader

Optimal Bipartite Consensus Control for Heterogeneous Unknown Multi-Agent Systems Via Reinforcement Learning

Flexible Formation Control Using Hausdorff Distance: A Multi-agent Reinforcement Learning Approach.