Abstract:Unmanned Aerial Vehicles (UAVs), also known as drones, have advanced greatly in recent years. There are many ways in which drones can be used, including transportation, photography, climate monitoring, and disaster relief. The reason for this is their high level of efficiency and safety in all operations. While the design of drones strives for perfection, it is not yet flawless. When it comes to detecting and preventing collisions, drones still face many challenges. In this context, this paper describes a methodology for developing a drone system that operates autonomously without the need for human intervention. This study applies reinforcement learning algorithms to train a drone to avoid obstacles autonomously in discrete and continuous action spaces based solely on image data. The novelty of this study lies in its comprehensive assessment of the advantages, limitations, and future research directions of obstacle detection and avoidance for drones, using different reinforcement learning techniques. This study compares three different reinforcement learning strategies—namely, Deep Q-Networks (DQN), Proximal Policy Optimization (PPO), and Soft Actor-Critic (SAC)—that can assist in avoiding obstacles, both stationary and moving; however, these strategies have been more successful in drones. The experiment has been carried out in a virtual environment made available by AirSim. Using Unreal Engine 4, the various training and testing scenarios were created for understanding and analyzing the behavior of RL algorithms for drones. According to the training results, SAC outperformed the other two algorithms. PPO was the least successful among the algorithms, indicating that on-policy algorithms are ineffective in extensive 3D environments with dynamic actors. DQN and SAC, two off-policy algorithms, produced encouraging outcomes. However, due to its constrained discrete action space, DQN may not be as advantageous as SAC in narrow pathways and twists. Concerning further findings, when it comes to autonomous drones, off-policy algorithms, such as DQN and SAC, perform more effectively than on-policy algorithms, such as PPO. The findings could have practical implications for the development of safer and more efficient drones in the future.

EPO-S: A Constrained RL Method to Enhance UAV Safety with Spatial Representation

Representation Enhancement-Based Proximal Policy Optimization for UAV Path Planning and Obstacle Avoidance

Relevant experience learning: A deep reinforcement learning method for UAV autonomous motion planning in complex unknown environments

Q Learning Algorithm Based UAV Path Learning and Obstacle Avoidence Approach

Path Planning Technology of Unmanned Vehicle Based on Improved Deep Reinforcement Learning

Reinforcement Learning-Based Low-Altitude Path Planning for UAS Swarm in Diverse Threat Environments

A Hybrid Human-in-the-Loop Deep Reinforcement Learning Method for UAV Motion Planning for Long Trajectories with Unpredictable Obstacles

A Modified Q-learning Algorithm Facing for Path Planning and Obstacle Avoidance Method for Unmanned Aerial Vehicle

Safety Constrained Trajectory Optimization for Completion Time Minimization for UAV Communications

A UAV Path Planning Method Based on Deep Reinforcement Learning

Path Planning for Unmanned Aerial Vehicle Via Off-Policy Reinforcement Learning with Enhanced Exploration

A Reinforcement Learning-based Decentralized Method of Avoiding Multi-UAV Collision in 3-D Airspace

Learning-Based UAV Coverage-Aware Path Planning in Large-scale Urban Environments

UAV Autonomous Navigation Based on Deep Reinforcement Learning in Highly Dynamic and High-Density Environments

Autonomous Navigation for Cellular-Connected UAV in Highly Dynamic Environments: A Deep Reinforcement Learning Approach

Intelligent Decision-Making for 3-Dimensional Dynamic Obstacle Avoidance of UAV Based on Deep Reinforcement Learning

Collision Avoidance for Multiple UAVs in Unknown Scenarios with Causal Representation Disentanglement

3M-RL: Multi-Resolution, Multi-Agent, Mean-Field Reinforcement Learning for Autonomous UAV Routing

Model-guided Reinforcement Learning Enclosing for UAVs with Collision-free and Reinforced Tracking Capability

Deep Reinforcement Learning for Vision-Based Navigation of UAVs in Avoiding Stationary and Mobile Obstacles

Rule-Guided DRL for UAV-Assisted Wireless Sensor Networks with No-Fly Zones Safety