Abstract:Unmanned aerial vehicles (UAV), also known as drones have been used for a variety of reasons and the commercial drone market growth is expected to reach remarkable levels in the near future. However, some drone users can mistakenly or intentionally fly into flight paths at major airports, flying too close to commercial aircraft or invading people's privacy. In order to prevent these unwanted events, counter-drone technology is needed to eliminate threats from drones and hopefully they can be integrated into the skies safely. There are various counter-drone methods available in the industry. However, a counter-drone system supported by an artificial intelligence (AI) method can be an efficient way to fight against drones instead of human intervention. In this paper, a deep reinforcement learning (DRL) method has been proposed to counter a drone in a 3D space by using another drone. In a 2D space it is already shown that the deep reinforcement learning method is an effective way to counter a drone. However, countering a drone in a 3D space with another drone is a very challenging task considering the time required to train and avoid obstacles at the same time. A Deep Q-Network (DQN) algorithm with dueling network architecture and prioritized experience replay is presented to catch another drone in the environment provided by an Airsim simulator. The models have been trained and tested with different scenarios to analyze the learning progress of the drone. Experiences from previous training are also transferred before starting a new training by pre-processing the previous experiences and eliminating those considered as bad experiences. The results show that the best models are obtained with transfer learning and the drone learning progress has been increased dramatically. Additionally, an algorithm which combines imitation learning and reinforcement learning is implemented to catch the target drone. In this algorithm, called deep q-learning from demonstrations (DQfD), expert demonstrations data and self-generated data by the agent are sampled and the agent continues learning without overwriting the demonstration data. The main advantage of this algorithm is to accelerate the learning process even if there is a small amount of demonstration data.

Canaries and Whistles: Resilient Drone Communication Networks with (or without) Deep Reinforcement Learning

Multi-Agent Reinforcement Learning Based UAV Swarm Communications Against Jamming

Safe Exploration in Wireless Security: A Safe Reinforcement Learning Algorithm With Hierarchical Structure

Safe Multi-Agent Reinforcement Learning for Wireless Applications Against Adversarial Communications

Multi-Agent Reinforcement Learning for Wireless Networks Against Adversarial Communications

QoS and Jamming-Aware Wireless Networking Using Deep Reinforcement Learning

Autonomous Network Defence using Reinforcement Learning

Deep Reinforcement Learning for Interference Management in UAV-based 3D Networks: Potentials and Challenges

Reinforcement learning-based autonomous attacker to uncover computer network vulnerabilities

Inroads into Autonomous Network Defence using Explained Reinforcement Learning

Countering a Drone in a 3D Space: Analyzing Deep Reinforcement Learning Methods

Distributed Drone Base Station Positioning for Emergency Cellular Networks Using Reinforcement Learning

A Deep Reinforcement Learning Approach to Efficient Drone Mobility Support

Application of Deep Reinforcement Learning to Defense and Intrusion Strategies Using Unmanned Aerial Vehicles in a Versus Game

Adversarial jamming attacks and defense strategies via adaptive deep reinforcement learning

Deployment of Unmanned Aerial Vehicles in Next-Generation Wireless Communication Network Using Multi-Agent Reinforcement Learning

Obstacle-Aware and Energy-Efficient Multi-Drone Coordination and Networking for Disaster Response

Multi-UAV Collaborative Surveillance Network Recovery Via Deep Reinforcement Learning

Intercepting Unauthorized Aerial Robots in Controlled Airspace Using Reinforcement Learning

AdaptNet: Rethinking Sensing and Communication for a Seamless Internet of Drones Experience

Energy-Aware Multi-Agent Reinforcement Learning for Collaborative Execution in Mission-Oriented Drone Networks