Abstract:Here, a novel hierarchical game theoretic algorithm has been developed to provide a feasible theory foundation for solving large‐scale multi‐agent system distributed optimal formation problems by effectively integrating the mean‐field game, the Stackelberg game, and the cooperative game. This algorithm addresses the heavy computational complexity and communication traffic challenges while directly extending conventional distributed control from small scale to large scale. Moreover, a hybrid actor–critic‐based reinforcement learning algorithm is constructed to learn the solution of the hierarchical game‐based optimal distributed formation control. This paper introduces a distributed adaptive formation control for large‐scale multi‐agent systems (LS‐MAS) that addresses the heavy computational complexity and communication traffic challenges while directly extending conventional distributed control from small scale to large scale. Specifically, a novel hierarchical game theoretic algorithm is developed to provide a feasible theory foundation for solving LS‐MAS distributed optimal formation problem by effectively integrating the mean‐field game (MFG), the Stackelberg game, and the cooperative game. In particular, LS‐MAS is divided into multiple groups geographically with each having one group leader and a significant amount of followers. Then, a cooperative game is used among multi‐group leaders to formulate distributed inter‐group formation control for leaders. Meanwhile, an MFG is adopted for a large number of intra‐group followers to achieve the collective intra‐group formation while a Stackelberg game is connecting the followers with their corresponding leader within the same group to achieve the overall LS‐MAS multi‐group formation behavior. Moreover, a hybrid actor–critic‐based reinforcement learning algorithm is constructed to learn the solution of the hierarchical game‐based optimal distributed formation control. Finally, to show the effectiveness of the presented schemes, numerical simulations and Lyapunov analysis is performed.

Zero-Sum Game-Based Controller Design Using Reinforcement Learning for Formation Tracking of Multi-agent Systems

Moving Forward in Formation: A Decentralized Hierarchical Learning Approach to Multi-Agent Moving Together

Group Formation Tracking of Heterogeneous Multi-Agent Systems Using Reinforcement Learning

Distributed output formation tracking control of heterogeneous multi-agent systems using reinforcement learning

Finite-horizon robust formation-containment control of multi-agent networks with unknown dynamics

Resilient Output Formation-Containment Tracking of Heterogeneous Multi-Agent Systems: A Learning-Based Framework using Dynamic Data

Leader-follower Formation Control for a Multi-missile System Via Deep Reinforcement Learning

Optimal Tracking Control of Nonlinear Multiagent Systems Using Internal Reinforce Q-Learning

Hierarchical game theoretical distributed adaptive control for large scale multi‐group multi‐agent system

Formation cooperative trajectory tracking control for unmanned aerial vehicles via differential game and reinforcement learning

Distributed Formation Control for a Class of Human-in-the-Loop Multiagent Systems

Optimal robust formation control for heterogeneous multi‐agent systems based on reinforcement learning

Behavior Learning Based Distributed Tracking Control for Human-in-the-Loop Multi-Agent Systems

Leader-Follower Formation Learning Control of Discrete-Time Nonlinear Multiagent Systems

Decentralized Optimal Tracking Control for Large-scale Multi-Agent Systems under Complex Environment: A Constrained Mean Field Game with Reinforcement Learning Approach

Flexible Formation Control Using Hausdorff Distance: A Multi-agent Reinforcement Learning Approach.

Relative Distributed Formation and Obstacle Avoidance with Multi-agent Reinforcement Learning

Heterogeneous formation control of multiple rotorcrafts with unknown dynamics by reinforcement learning

Adaptive Event-Triggered Bipartite Formation for Multiagent Systems via Reinforcement Learning

Decentralized Multi-Robot Formation Control Using Reinforcement Learning

A Q-Learning-Based Parameters Adaptive Algorithm for Formation Tracking Control of Multi-Mobile Robot Systems