Abstract:Most existing multi-UAV collaborative search methods only consider scenarios of two-dimensional path planning or static target search. To be close to the practical scenario, this paper proposes a path planning method based on an action-mask-based multi-agent proximal policy optimization (AM-MAPPO) algorithm for multiple UAVs searching for moving targets in three-dimensional (3D) environments. In particular, a multi-UAV high–low altitude collaborative search architecture is introduced that not only takes into account the extensive detection range of high-altitude UAVs but also leverages the benefit of the superior detection quality of low-altitude UAVs. The optimization objective of the search task is to minimize the uncertainty of the search area while maximizing the number of captured moving targets. The path planning problem for moving target search in a 3D environment is formulated and addressed using the AM-MAPPO algorithm. The proposed method incorporates a state representation mechanism based on field-of-view encoding to handle dynamic changes in neural network input dimensions and develops a rule-based target capture mechanism and an action-mask-based collision avoidance mechanism to enhance the AM-MAPPO algorithm’s convergence speed. Experimental results demonstrate that the proposed algorithm significantly reduces regional uncertainty and increases the number of captured moving targets compared to other deep reinforcement learning methods. Ablation studies further indicate that the proposed action mask mechanism, target capture mechanism, and collision avoidance mechanism of the AM-MAPPO algorithm can improve the algorithm’s effectiveness, target capture capability, and UAVs’ safety, respectively.

Multi-UAV Collaborative Search and Strike based on Reinforcement Learning

UAV Cooperative Search Based on Multi-agent Generative Adversarial Imitation Learning

Multi-UAV Adaptive Cooperative Formation Trajectory Planning Based on an Improved MATD3 Algorithm of Deep Reinforcement Learning

Multi-UAV Autonomous Collaborative Target Tracking in Dynamic Environment Based on Multi-agent Reinforcement Learning

Collaborative Decision-Making Method for Multi-UAV Based on Multiagent Reinforcement Learning

UAV Swarm Cooperative Target Search: A Multi-Agent Reinforcement Learning Approach

Reinforcement-Learning-Based Multi-UAV Cooperative Search for Moving Targets in 3D Scenarios

Multi-Agent Cooperative Target Search

Multi-UAVs distributed cooperative target search algorithm with controllable revisit mechanism based on digital pheromone

Searching and Tracking an Unknown Number of Targets: A Learning-Based Method Enhanced with Maps Merging

Multi-UAV Cooperative Short-Range Combat Via Attention-Based Reinforcement Learning Using Individual Reward Shaping

Multi-UAV Cooperative Target Tracking Based on Swarm Intelligence

A Method of Multi-UAV Cooperative Task Assignment Based on Reinforcement Learning

Multi-UAV Collaborative Detection Based on Reinforcement Learning.

Cooperatively Pursuing a Target Unmanned Aerial Vehicle by Multiple Unmanned Aerial Vehicles Based on Multiagent Reinforcement Learning

Multi-UAV Collaborative Target Search Method in Unknown Dynamic Environment

Collaborative Search Planning of UAV Swarms Based on Deep Reinforcement Learning

Multi-UAV Cooperative Search in Multi-Layered Aerial Computing Networks: A Multi-Agent Deep Reinforcement Learning Approach

A Reinforcement Learning Approach for UAV Target Searching and Tracking

Multi-UAV Distributed Collaborative Coverage for Target Search Using Heuristic Strategy

Multi-UAVs Cooperative Target Search with Communication Constraint