Abstract:Unmanned aerial vehicles (UAV), also known as drones have been used for a variety of reasons and the commercial drone market growth is expected to reach remarkable levels in the near future. However, some drone users can mistakenly or intentionally fly into flight paths at major airports, flying too close to commercial aircraft or invading people's privacy. In order to prevent these unwanted events, counter-drone technology is needed to eliminate threats from drones and hopefully they can be integrated into the skies safely. There are various counter-drone methods available in the industry. However, a counter-drone system supported by an artificial intelligence (AI) method can be an efficient way to fight against drones instead of human intervention. In this paper, a deep reinforcement learning (DRL) method has been proposed to counter a drone in a 3D space by using another drone. In a 2D space it is already shown that the deep reinforcement learning method is an effective way to counter a drone. However, countering a drone in a 3D space with another drone is a very challenging task considering the time required to train and avoid obstacles at the same time. A Deep Q-Network (DQN) algorithm with dueling network architecture and prioritized experience replay is presented to catch another drone in the environment provided by an Airsim simulator. The models have been trained and tested with different scenarios to analyze the learning progress of the drone. Experiences from previous training are also transferred before starting a new training by pre-processing the previous experiences and eliminating those considered as bad experiences. The results show that the best models are obtained with transfer learning and the drone learning progress has been increased dramatically. Additionally, an algorithm which combines imitation learning and reinforcement learning is implemented to catch the target drone. In this algorithm, called deep q-learning from demonstrations (DQfD), expert demonstrations data and self-generated data by the agent are sampled and the agent continues learning without overwriting the demonstration data. The main advantage of this algorithm is to accelerate the learning process even if there is a small amount of demonstration data.

Reinforcement Learning-Based Counter Fixed-Wing Drone System Using GNSS Deception

Countering a Drone in a 3D Space: Analyzing Deep Reinforcement Learning Methods

Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning

Intercepting Unauthorized Aerial Robots in Controlled Airspace Using Reinforcement Learning

Application of Deep Reinforcement Learning to Defense and Intrusion Strategies Using Unmanned Aerial Vehicles in a Versus Game

Chasing the Intruder: A Reinforcement Learning Approach for Tracking Intruder Drones

Decentralized Reinforcement Learning for Multi-Target Search and Detection by a Team of Drones

UAV Multi-Dynamic Target Interception: A Hybrid Intelligent Method Using Deep Reinforcement Learning and Fuzzy Logic

Deep Reinforcement Learning-Driven Reconfigurable Intelligent Surface-Assisted Radio Surveillance with a Fixed-Wing UAV.

Deep Reinforcement Learning for Vision-Based Navigation of UAVs in Avoiding Stationary and Mobile Obstacles

Secure Control Systems for Autonomous Quadrotors against Cyber-Attacks

Counterfactual Threshold Learning for Drone RF Signal Classification under Interference Conditions

D3GF: A Study on Optimal Defense Performance Evaluation of Drone-Type Moving Target Defense Through Game Theory

UAV Detection Using Reinforcement Learning

RIS-Assisted Jamming Rejection and Path Planning for UAV-Borne IoT Platform: A New Deep Reinforcement Learning Framework

An Application of Inverse Reinforcement Learning to Estimate Interference in Drone Swarms

Robust Adversarial Attacks Detection based on Explainable Deep Reinforcement Learning For UAV Guidance and Planning

Secure Tracking Control via Fixed-Time Convergent Reinforcement Learning for a UAV CPS

Multi-Agent System for Rogue Drone Interception

UAV-Enabled Secure Communications by Multi-Agent Deep Reinforcement Learning

Gradient Monitored Reinforcement Learning for Jamming Attack Detection in FANETs