Abstract:Unmanned aerial vehicles (UAV), also known as drones have been used for a variety of reasons and the commercial drone market growth is expected to reach remarkable levels in the near future. However, some drone users can mistakenly or intentionally fly into flight paths at major airports, flying too close to commercial aircraft or invading people's privacy. In order to prevent these unwanted events, counter-drone technology is needed to eliminate threats from drones and hopefully they can be integrated into the skies safely. There are various counter-drone methods available in the industry. However, a counter-drone system supported by an artificial intelligence (AI) method can be an efficient way to fight against drones instead of human intervention. In this paper, a deep reinforcement learning (DRL) method has been proposed to counter a drone in a 3D space by using another drone. In a 2D space it is already shown that the deep reinforcement learning method is an effective way to counter a drone. However, countering a drone in a 3D space with another drone is a very challenging task considering the time required to train and avoid obstacles at the same time. A Deep Q-Network (DQN) algorithm with dueling network architecture and prioritized experience replay is presented to catch another drone in the environment provided by an Airsim simulator. The models have been trained and tested with different scenarios to analyze the learning progress of the drone. Experiences from previous training are also transferred before starting a new training by pre-processing the previous experiences and eliminating those considered as bad experiences. The results show that the best models are obtained with transfer learning and the drone learning progress has been increased dramatically. Additionally, an algorithm which combines imitation learning and reinforcement learning is implemented to catch the target drone. In this algorithm, called deep q-learning from demonstrations (DQfD), expert demonstrations data and self-generated data by the agent are sampled and the agent continues learning without overwriting the demonstration data. The main advantage of this algorithm is to accelerate the learning process even if there is a small amount of demonstration data.

Efficient deep reinforcement learning under task variations via knowledge transfer for drone control

Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

Task and Domain Adaptive Reinforcement Learning for Robot Control

Energy-Efficient Online Path Planning of Multiple Drones Using Reinforcement Learning

DL-DRL: A double-level deep reinforcement learning approach for large-scale task scheduling of multi-UAV

Autonomous Navigation via Deep Reinforcement Learning for Resource Constraint Edge Nodes Using Transfer Learning

Learning Agile Flights through Narrow Gaps with Varying Angles using Onboard Sensing

Continuous Transfer Learning for UAV Communication-aware Trajectory Design

Dexterous robotic manipulation using deep reinforcement learning and knowledge transfer for complex sparse reward‐based tasks

Improving Generalization in Aerial and Terrestrial Mobile Robots Control Through Delayed Policy Learning

Countering a Drone in a 3D Space: Analyzing Deep Reinforcement Learning Methods

Joint Speed Control and Energy Replenishment Optimization for UAV-assisted IoT Data Collection with Deep Reinforcement Transfer Learning

Deep Reinforcement Learning Multi-UAV Trajectory Control for Target Tracking

Asynchronous Curriculum Experience Replay: A Deep Reinforcement Learning Approach for UAV Autonomous Motion Control in Unknown Dynamic Environments

DGTRL: Deep graph transfer reinforcement learning method based on fusion of knowledge and data

Digital Twin Assisted Task Assignment in Multi-UAV Systems: A Deep Reinforcement Learning Approach

3D-Trajectory and Phase-Shift Design for RIS-Assisted UAV Systems Using Deep Reinforcement Learning

Autonomous obstacle avoidance of UAV based on deep reinforcement learning

Deep reinforcement learning for unmanned aerial vehicles cluster task allocation

A data-efficient goal-directed deep reinforcement learning method for robot visuomotor skill

Deep Reinforcement Learning-Based End-to-End Control for UAV Dynamic Target Tracking