Abstract:Here, a novel hierarchical game theoretic algorithm has been developed to provide a feasible theory foundation for solving large‐scale multi‐agent system distributed optimal formation problems by effectively integrating the mean‐field game, the Stackelberg game, and the cooperative game. This algorithm addresses the heavy computational complexity and communication traffic challenges while directly extending conventional distributed control from small scale to large scale. Moreover, a hybrid actor–critic‐based reinforcement learning algorithm is constructed to learn the solution of the hierarchical game‐based optimal distributed formation control. This paper introduces a distributed adaptive formation control for large‐scale multi‐agent systems (LS‐MAS) that addresses the heavy computational complexity and communication traffic challenges while directly extending conventional distributed control from small scale to large scale. Specifically, a novel hierarchical game theoretic algorithm is developed to provide a feasible theory foundation for solving LS‐MAS distributed optimal formation problem by effectively integrating the mean‐field game (MFG), the Stackelberg game, and the cooperative game. In particular, LS‐MAS is divided into multiple groups geographically with each having one group leader and a significant amount of followers. Then, a cooperative game is used among multi‐group leaders to formulate distributed inter‐group formation control for leaders. Meanwhile, an MFG is adopted for a large number of intra‐group followers to achieve the collective intra‐group formation while a Stackelberg game is connecting the followers with their corresponding leader within the same group to achieve the overall LS‐MAS multi‐group formation behavior. Moreover, a hybrid actor–critic‐based reinforcement learning algorithm is constructed to learn the solution of the hierarchical game‐based optimal distributed formation control. Finally, to show the effectiveness of the presented schemes, numerical simulations and Lyapunov analysis is performed.

Multi-agent Coordination of Networked Mobile Agents with Hierarchical Dynamic Graph

Distributed Adaptive Cooperative Control for a Class of Nonlinear Multi-Agent Systems Via Fully Event-triggered Mechanism

Learning Intra-group Cooperation in Multi-agent Systems.

Moving Forward in Formation: A Decentralized Hierarchical Learning Approach to Multi-Agent Moving Together

Redundancy-Reduction-Based Hierarchical Design in Synchronization of Multi-Agent Systems

Coordination Control for a Class of Multi-Agent Systems under Asynchronous Switching

Collective Behavior for Group of Generic Linear Agents Interacting under Arbitrary Network Topology

Entropy Enhanced Multi-Agent Coordination Based on Hierarchical Graph Learning for Continuous Action Space

Coordination of multiagents interacting under independent position and velocity topologies.

Scalable and Transferable Reinforcement Learning for Multi-Agent Mixed Cooperative–Competitive Environments Based on Hierarchical Graph Attention

Federated Control with Hierarchical Multi-Agent Deep Reinforcement Learning

Multi-agent Coordination Under Temporal Logic Tasks and Team-Wise Intermittent Communication

Dynamic behavior of discrete-time multiagent systems with general communication structures

Flocking Coordination Of Multiple Interactive Dynamical Agents With Switching Topology

Navigation Function Based Decentralized Control of A Multi-Agent System with Network Connectivity Constraints

Controllability of Heterogeneous Multi-Agent Systems

An Examination of Coordination for Homogeneous Linear Agents under Arbitrary Network Topology

A hierarchical design framework for distributed control of multi-agent systems

Hierarchical game theoretical distributed adaptive control for large scale multi‐group multi‐agent system

Distributed Circle Formation Control over Directed Networks with Communication Constraints

Coordination for Linear Multiagent Systems with Dynamic Interaction Topology in the Leader-Following Framework