Abstract:Purpose This paper aims to realize a fully distributed multi-UAV collision detection and avoidance based on deep reinforcement learning (DRL). To deal with the problem of low sample efficiency in DRL and speed up the training. To improve the applicability and reliability of the DRL-based approach in multi-UAV control problems. Design/methodology/approach In this paper, a fully distributed collision detection and avoidance approach for multi-UAV based on DRL is proposed. A method that integrates human experience into policy training via a human experience-based adviser is proposed. The authors propose a hybrid control method which combines the learning-based policy with traditional model-based control. Extensive experiments including simulations, real flights and comparative experiments are conducted to evaluate the performance of the approach. Findings A fully distributed multi-UAV collision detection and avoidance method based on DRL is realized. The reward curve shows that the training process when integrating human experience is significantly accelerated and the mean episode reward is higher than the pure DRL method. The experimental results show that the DRL method with human experience integration has a significant improvement than the pure DRL method for multi-UAV collision detection and avoidance. Moreover, the safer flight brought by the hybrid control method has also been validated. Originality/value The fully distributed architecture is suitable for large-scale unmanned aerial vehicle (UAV) swarms and real applications. The DRL method with human experience integration has significantly accelerated the training compared to the pure DRL method. The proposed hybrid control strategy makes up for the shortcomings of two-dimensional light detection and ranging and other puzzles in applications.

Study on the resolution of multi-aircraft flight conflicts based on an IDQN

Priority-based intelligent resolution method of multi-aircraft flight conflicts

General real-time three-dimensional multi-aircraft conflict resolution method using multi-agent reinforcement learning

Spacecraft Attitude Maneuver Planning Based on Deep Reinforcement Learning under Complex Constraints

Review of Deep Reinforcement Learning Approaches for Conflict Resolution in Air Traffic Control

General multi-agent reinforcement learning integrating adaptive manoeuvre strategy for real-time multi-aircraft conflict resolution

Three-Dimensional Aircraft Conflict Resolution Based on Smoothing Methods

Enhancing Air Traffic Control: A Transparent Deep Reinforcement Learning Framework for Autonomous Conflict Resolution

Autonomous air traffic separation assurance through machine learning

Solving the Problem of Detecting and Resolving Conflicts in Airspace Using Deep Reinforcement Learning

A Deep Multi-Agent Reinforcement Learning Approach to Autonomous Separation Assurance

Deep Reinforcement Learning With Application to Air Confrontation Intelligent Decision-Making of Manned/Unmanned Aerial Vehicle Cooperative System

Autonomous maneuver decision-making for a UCAV in short-range aerial combat based on an MS-DDQN algorithm

An Effective and Scalable Approach for Swarm-on-Swarm Air Combat Decision

Autonomous Air Traffic Controller: A Deep Multi-Agent Reinforcement Learning Approach

Integrating human experience in deep reinforcement learning for multi-UAV collision detection and avoidance

Autonomous Conflict Resolution Method for Multi-Uavs Based on Preorder Flight Information

Reinforcement Learning-Based Air Traffic Deconfliction

Multi-intent autonomous decision-making for air combat with deep reinforcement learning

Markov Decision Process-Based Distributed Conflict Resolution for Drone Air Traffic Management

Digital Twin Assisted Task Assignment in Multi-UAV Systems: A Deep Reinforcement Learning Approach