Abstract:Multiagent reinforcement learning (MARL) algorithms have shown promise in the Internet of Things devices, such as unmanned aerial vehicle (UAV) swarms. However, the dynamic nature of large-scale swarm systems, with constantly changing numbers of agents and observed neighbors, poses challenges for MARL adaptation. Existing approaches struggle to extract meaningful features and require a substantial number of experience samples, resulting in low-sample efficiency and high-risk ratios. Moreover, these methods are effective in task-specific scenarios and fail to perform well in multitask settings. To overcome these challenges, this study proposes a high-sample efficient and scalable MARL approach for UAV swarms. The proposed approach incorporates a hypernetwork-based embedding attention (HEA) mechanism for the state representation of the policy network and a multiencoder gated transformer with a multilayer attention (MEGTrMA) mechanism for the value function. The HEA automatically generates weights for each agent to adapt to dynamic scenarios, enhancing representation ability and adaptability while reducing the cost of trial and error for improved learning efficiency. The MEGTrMA captures the contribution of each agent to the global observation, establishing long-term dependencies among them and facilitating stable policy learning in multitask scenarios. Simulation results demonstrate that the proposed method is scalable, generalizable, and high-sample efficient. Compared to learning from scratch, our method significantly reduces training time to less than one-fifth of the initial time by progressively increasing the number of UAVs and their corresponding neighbors. Additionally, the average number of collisions is reduced by an order of magnitude for large-scale UAV swarms.

Oracle-Guided Deep Reinforcement Learning for Large-Scale Multi-UAVs Flocking and Navigation.

An Improved Method towards Multi-UAV Autonomous Navigation Using Deep Reinforcement Learning

High-Sample-Efficient Multiagent Reinforcement Learning for Navigation and Collision Avoidance of UAV Swarms in Multitask Environments

Multi-Agent Deep Reinforcement Learning for UAVs Navigation in Unknown Complex Environment

Flocking Control of UAV Swarms with Deep Reinforcement Leaming Approach

Multi-UAV Adaptive Cooperative Formation Trajectory Planning Based on an Improved MATD3 Algorithm of Deep Reinforcement Learning

Learning-Based Multi-UAV Flocking Control With Limited Visual Field and Instinctive Repulsion

Graph-Based Multi-agent Reinforcement Learning for Large-Scale UAVs Swarm System Control

Autonomous Navigation of UAVs in Large-Scale Complex Environments: A Deep Reinforcement Learning Approach

Autonomous Navigation of UAV in Large-Scale Unknown Complex Environment with Deep Reinforcement Learning.

Collaborative Decision-Making Method for Multi-UAV Based on Multiagent Reinforcement Learning

UAV Swarm Confrontation Using Hierarchical Multiagent Reinforcement Learning

Multi-UAV Autonomous Collaborative Target Tracking in Dynamic Environment Based on Multi-agent Reinforcement Learning

Deep Reinforcement Learning for Flocking Motion of Multi-UAV systems: Learn from a Digital Twin

PASCAL: PopulAtion-Specific Curriculum-based MADRL for collision-free flocking with large-scale fixed-wing UAV swarms

Attention-Based Population-Invariant Deep Reinforcement Learning for Collision-Free Flocking with A Scalable Fixed-Wing UAV Swarm.

Application of Deep Reinforcement Learning to UAV Swarming for Ground Surveillance

Deep-Reinforcement-Learning-Based Autonomous UAV Navigation With Sparse Rewards

Distributed Energy-Efficient Multi-UAV Navigation for Long-Term Communication Coverage by Deep Reinforcement Learning

Multi-UAV Path Planning and Following Based on Multi-Agent Reinforcement Learning

Maximizing UAV Coverage in Maritime Wireless Networks: A Multiagent Reinforcement Learning Approach