Abstract:Unmanned aerial vehicles (UAV), also known as drones have been used for a variety of reasons and the commercial drone market growth is expected to reach remarkable levels in the near future. However, some drone users can mistakenly or intentionally fly into flight paths at major airports, flying too close to commercial aircraft or invading people's privacy. In order to prevent these unwanted events, counter-drone technology is needed to eliminate threats from drones and hopefully they can be integrated into the skies safely. There are various counter-drone methods available in the industry. However, a counter-drone system supported by an artificial intelligence (AI) method can be an efficient way to fight against drones instead of human intervention. In this paper, a deep reinforcement learning (DRL) method has been proposed to counter a drone in a 3D space by using another drone. In a 2D space it is already shown that the deep reinforcement learning method is an effective way to counter a drone. However, countering a drone in a 3D space with another drone is a very challenging task considering the time required to train and avoid obstacles at the same time. A Deep Q-Network (DQN) algorithm with dueling network architecture and prioritized experience replay is presented to catch another drone in the environment provided by an Airsim simulator. The models have been trained and tested with different scenarios to analyze the learning progress of the drone. Experiences from previous training are also transferred before starting a new training by pre-processing the previous experiences and eliminating those considered as bad experiences. The results show that the best models are obtained with transfer learning and the drone learning progress has been increased dramatically. Additionally, an algorithm which combines imitation learning and reinforcement learning is implemented to catch the target drone. In this algorithm, called deep q-learning from demonstrations (DQfD), expert demonstrations data and self-generated data by the agent are sampled and the agent continues learning without overwriting the demonstration data. The main advantage of this algorithm is to accelerate the learning process even if there is a small amount of demonstration data.

Distributed Deep Reinforcement Learning for Drone Swarm Control

Decentralized Control of Quadrotor Swarms with End-to-end Deep Reinforcement Learning

Learning Vision-Based Flight in Drone Swarms by Imitation

Application of Deep Reinforcement Learning to UAV Swarming for Ground Surveillance

Federated Reinforcement Learning for Collective Navigation of Robotic Swarms

Learning to Navigate in Turbulent Flows with Aerial Robot Swarms: A Cooperative Deep Reinforcement Learning Approach

Learning Vision-based Cohesive Flight in Drone Swarms

Decentralized Reinforcement Learning for Multi-Target Search and Detection by a Team of Drones

Continuous Deep Hierarchical Reinforcement Learning for Ground-Air Swarm Shepherding

Drone swarm patrolling with uneven coverage requirements

Scalable and Cooperative Deep Reinforcement Learning Approaches for Multi-UAV Systems: A Systematic Review

Countering a Drone in a 3D Space: Analyzing Deep Reinforcement Learning Methods

Collision Avoidance and Navigation for a Quadrotor Swarm Using End-to-end Deep Reinforcement Learning

Multi-target tracking for unmanned aerial vehicle swarms using deep reinforcement learning

Learning-Based Multi-UAV Flocking Control With Limited Visual Field and Instinctive Repulsion

Short vs. Long-term Coordination of Drones: When Distributed Optimization Meets Deep Reinforcement Learning

Deep Reinforcement Learning Based Three-Dimensional Area Coverage With UAV Swarm

Distributed Machine Learning for UAV Swarms: Computing, Sensing, and Semantics

Neural-Swarm: Decentralized Close-Proximity Multirotor Control Using Learned Interactions

VGAI: End-to-End Learning of Vision-Based Decentralized Controllers for Robot Swarms

Decentralized Multi-Robot Formation Control Using Reinforcement Learning