Abstract:Multi-unmanned aerial vehicle (multi-UAV) cooperative trajectory planning is an extremely challenging issue in UAV research field due to its NP-hard characteristic, collision avoiding constraints, close formation requirement, consensus convergence and high-dimensional action space etc. Especially, the difficulty of multi-UAV trajectory planning will boost comparatively when there are complex obstacles and narrow passages in unknown environments. Accordingly, a novel multi-UAV adaptive cooperative formation trajectory planning approach is proposed in this paper based on an improved deep reinforcement learning algorithm in unknown obstacle environments, which innovatively introduces long short-term memory (LSTM) recurrent neural network (RNN) into the environment perception end of multiagent twin delayed deep deterministic policy gradient (MATD3) network, and develops an improved potential field-based dense reward function to strengthen the policy learning efficiency and accelerates the convergence respectively. Moreover, a hierarchical deep reinforcement learning training mechanism, including adaptive formation layer, trajectory planning layer and action execution layer is implemented to explore an optimal trajectory planning policy. Additionally, an adaptive formation maintaining and transformation strategy is presented for UAV swarm to comply with the environment with narrow passages. Simulation results show that the proposed approach is better in policy learning efficiency, optimality of trajectory planning policy and adaptability to narrow passages than that using multi-agent deep deterministic policy gradient (MADDPG) and MATD3.

Enhanced Multi-Agent Proximal Policy Optimization for Multi-UAV Target Offensive-Defensive Decision

Mean policy-based proximal policy optimization for maneuvering decision in multi-UAV air combat

UAV Cooperative Air Combat Maneuvering Confrontation Based on Multi-agent Reinforcement Learning

Research on the Multiagent Joint Proximal Policy Optimization Algorithm Controlling Cooperative Fixed-Wing UAV Obstacle Avoidance

Research on multi-UAV task decision-making based on improved MADDPG algorithm and transfer learning

Multi-UAV Cooperative Air Combat Decision-Making Based on Multi-Agent Double-Soft Actor-Critic

Collaborative Decision-making in Heterogeneous UAV Swarms Based on Multi-agent Deep Reinforcement Learning

A Multi-agent Deep Reinforcement Learning Method for UAVs Cooperative Pursuit Problem

UAV Cooperative Air Combat Maneuvering Decision-Making Using GRU-MAPPO

Multi-UAV Adaptive Cooperative Formation Trajectory Planning Based on an Improved MATD3 Algorithm of Deep Reinforcement Learning

MW-MADDPG: a meta-learning based decision-making method for collaborative UAV swarm

Collaborative Decision-Making Method for Multi-UAV Based on Multiagent Reinforcement Learning

Multi-UAV Cooperative Search in Multi-Layered Aerial Computing Networks: A Multi-Agent Deep Reinforcement Learning Approach

Multi-UAV Cooperative Maneuver Decision-Making for Pursuit-Evasion Using Improved MADRL

UAV cooperative air combat maneuver decision based on multi-agent reinforcement learning

A Method of Multi-UAV Cooperative Task Assignment Based on Reinforcement Learning

Game of Drones: Intelligent Online Decision Making of Multi-UAV Confrontation

UAV-enabled Collaborative Beamforming via Multi-Agent Deep Reinforcement Learning

Multiple unmanned aerial vehicle coordinated strikes against ground targets based on an improved multi-agent deep deterministic policy gradient algorithm

DTPPO: Dual-Transformer Encoder-based Proximal Policy Optimization for Multi-UAV Navigation in Unseen Complex Environments

Multi-Agent Confrontation Game Based on Multi-Agent Reinforcement Learning