Abstract:Unmanned aerial vehicles (UAV), also known as drones have been used for a variety of reasons and the commercial drone market growth is expected to reach remarkable levels in the near future. However, some drone users can mistakenly or intentionally fly into flight paths at major airports, flying too close to commercial aircraft or invading people's privacy. In order to prevent these unwanted events, counter-drone technology is needed to eliminate threats from drones and hopefully they can be integrated into the skies safely. There are various counter-drone methods available in the industry. However, a counter-drone system supported by an artificial intelligence (AI) method can be an efficient way to fight against drones instead of human intervention. In this paper, a deep reinforcement learning (DRL) method has been proposed to counter a drone in a 3D space by using another drone. In a 2D space it is already shown that the deep reinforcement learning method is an effective way to counter a drone. However, countering a drone in a 3D space with another drone is a very challenging task considering the time required to train and avoid obstacles at the same time. A Deep Q-Network (DQN) algorithm with dueling network architecture and prioritized experience replay is presented to catch another drone in the environment provided by an Airsim simulator. The models have been trained and tested with different scenarios to analyze the learning progress of the drone. Experiences from previous training are also transferred before starting a new training by pre-processing the previous experiences and eliminating those considered as bad experiences. The results show that the best models are obtained with transfer learning and the drone learning progress has been increased dramatically. Additionally, an algorithm which combines imitation learning and reinforcement learning is implemented to catch the target drone. In this algorithm, called deep q-learning from demonstrations (DQfD), expert demonstrations data and self-generated data by the agent are sampled and the agent continues learning without overwriting the demonstration data. The main advantage of this algorithm is to accelerate the learning process even if there is a small amount of demonstration data.

Drone's Objective Inference Using Policy Error Inverse Reinforcement Learning

An Application of Inverse Reinforcement Learning to Estimate Interference in Drone Swarms

AirPilot: Interpretable PPO-based DRL Auto-Tuned Nonlinear PID Drone Controller for Robust Autonomous Flights

Intercepting Unauthorized Aerial Robots in Controlled Airspace Using Reinforcement Learning

Countering a Drone in a 3D Space: Analyzing Deep Reinforcement Learning Methods

Chasing the Intruder: A Reinforcement Learning Approach for Tracking Intruder Drones

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

Long-Term Planning with Deep Reinforcement Learning on Autonomous Drones

Robust Control Strategy for Quadrotor Drone Using Reference Model-Based Deep Deterministic Policy Gradient

Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone Racing

Autonomous Single-Image Drone Exploration With Deep Reinforcement Learning and Mixed Reality

Reservoir Computing for Drone Trajectory Intent Prediction: A Physics Informed Approach

Model-Free Guidance Method for Drones in Complex Environments Using Direct Policy Exploration and Optimization

Deep Reinforcement Learning for Vision-Based Navigation of UAVs in Avoiding Stationary and Mobile Obstacles

Learning Agile Flights through Narrow Gaps with Varying Angles using Onboard Sensing

Beyond Static Obstacles: Integrating Kalman Filter with Reinforcement Learning for Drone Navigation

Analyzing deep reinforcement learning model decisions with Shapley additive explanations for counter drone operations

UAV Multi-Dynamic Target Interception: A Hybrid Intelligent Method Using Deep Reinforcement Learning and Fuzzy Logic

LiDAR-based drone navigation with reinforcement learning

Control of an Autonomous Unmanned Aerial Vehicle Using Reinforcement Learning