Abstract:Target searching is crucial in real-world scenarios such as search and rescue in disaster sites and battlefield target reconnaissance. Unmanned aerial vehicles (UAVs) are an ideal technical solution for target searching in large-scale and high-risk areas because they are agile, low cost, and able to collaborate and carry different sensors. In complex scenarios like battlefields, due to the lack of communication infrastructures and the intensive interference, UAVs often operate in communication denied environments. As a result, fast and reliable communication channels between UAVs and ground operators are difficult to establish. Thus, in such conditions, UAVs must be able to complete tasks autonomously and intelligently, without receiving real-time commands from the operators. With the rapid advances in artificial intelligence, reinforcement learning has shown potentiality for solving continuous decision problems. The target searching problem studied in this paper falls into this category and is suitable for adopting reinforcement learning technologies. However, the feasibility of reinforcement learning in UAV-based target searching in communication denied environments is not clear and, thus, requires in-depth investigations. As a pilot study in this direction, this paper models the target searching problem in communication denied and confrontation situations and proposes a simulation environment based on this model. Extensive experiments are conducted to answer the following questions. (1) Can reinforcement learning be applied in target searching by multi-UAVs in communication denied environments? (2) What are the advantages and disadvantages of different reinforcement learning algorithms in solving this problem? (3) How the degree of communication denial influences the performance of these algorithms? The current mainstream reinforcement learning technologies are adopted to perform simulations, whose results are analyzed quantitatively, leading to the following observations. (1) Reinforcement learning can effectively solve target searching problems for multi-UAVs in communication denied environments. (2) Compared with other algorithms, an autonomous decision-making UAV cluster based on a deep Q-network (DQN)exhibits the best problem-solving ability. (3) The algorithm performance changes with the degree of communication denial but remains largely stable when the communication condition varies.

Reinforcement Learning-Based Drone Simulators: Survey, Practice, and Challenge

Countering a Drone in a 3D Space: Analyzing Deep Reinforcement Learning Methods

Drone Deep Reinforcement Learning: A Review

Vision Based Drone Obstacle Avoidance by Deep Reinforcement Learning

Air Learning: A Deep Reinforcement Learning Gym for Autonomous Aerial Robot Visual Navigation

Deep Reinforcement Learning for Vision-Based Navigation of UAVs in Avoiding Stationary and Mobile Obstacles

Autonomous Single-Image Drone Exploration With Deep Reinforcement Learning and Mixed Reality

A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Platform

Autonomous Multi-Drone Racing Method Based on Deep Reinforcement Learning

Evaluation of Reinforcement and Deep Learning Algorithms in Controlling Unmanned Aerial Vehicles

Application of Deep Reinforcement Learning in UAVs: A Review

Feasibility of reinforcement learning for UAV-based target searching in a simulated communication denied environment

Application of Deep Reinforcement Learning to Defense and Intrusion Strategies Using Unmanned Aerial Vehicles in a Versus Game

Deep Reinforcement Learning-based Quadcopter Controller: A Practical Approach and Experiments

Autonomous Drone Racing: A Survey

Deep Learning and Reinforcement Learning for Autonomous Unmanned Aerial Systems: Roadmap for Theory to Deployment

Autonomous Unmanned Aerial Vehicle navigation using Reinforcement Learning: A systematic review

Survey of Simulators for Aerial Robots

Vision-based Learning for Drones: A Survey

OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Control