Optimizing Search and Rescue UAV Connectivity in Challenging Terrain through Multi Q-Learning

Mohammed M. H. Qazzaz,Syed A. R. Zaidi,Desmond C. McLernon,Abdelaziz Salama,Aubida A. Al-Hameed

2024-05-16

Abstract:Using Unmanned Aerial Vehicles (UAVs) in Search and rescue operations (SAR) to navigate challenging terrain while maintaining reliable communication with the cellular network is a promising approach. This paper suggests a novel technique employing a reinforcement learning multi Q-learning algorithm to optimize UAV connectivity in such scenarios. We introduce a Strategic Planning Agent for efficient path planning and collision awareness and a Real-time Adaptive Agent to maintain optimal connection with the cellular base station. The agents trained in a simulated environment using multi Q-learning, encouraging them to learn from experience and adjust their decision-making to diverse terrain complexities and communication scenarios. Evaluation results reveal the significance of the approach, highlighting successful navigation in environments with varying obstacle densities and the ability to perform optimal connectivity using different frequency bands. This work paves the way for enhanced UAV autonomy and enhanced communication reliability in search and rescue operations.

Robotics

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper primarily addresses the issue of optimizing communication connectivity for Unmanned Aerial Vehicles (UAVs) during Search and Rescue (SAR) missions in complex terrains. Specifically: 1. **Background**: - UAVs are becoming increasingly important in SAR missions due to their rapid mobility, autonomous functions, and cost-effectiveness. - However, ensuring reliable communication between UAVs and ground stations in complex terrains poses significant challenges. 2. **Problems with existing methods**: - Traditional methods such as satellite communication have high latency and high costs. - Dedicated UAV communication channels require substantial infrastructure investment, making widespread application difficult. - While using existing cellular networks is feasible, path planning and communication optimization in complex terrains remain major challenges. 3. **Research objectives**: - Propose a new framework based on a multi-Q learning algorithm to optimize SAR UAV connectivity in complex terrains. - Dynamically adjust UAV trajectories through reinforcement learning to ensure seamless communication with ground stations in complex terrains. - Achieve the shortest path while ensuring communication quality, thereby improving the efficiency and safety of SAR missions. ### Summary This paper aims to solve the communication connectivity issue for UAVs during SAR missions in complex terrains by introducing a multi-Q learning algorithm, thereby enhancing the autonomy and communication reliability of UAVs in these missions.

Optimizing Search and Rescue UAV Connectivity in Challenging Terrain through Multi Q-Learning

An Adaptive Conversion Speed Q-Learning Algorithm for Search and Rescue UAV Path Planning in Unknown Environments

Deployment of Unmanned Aerial Vehicles in Next-Generation Wireless Communication Network Using Multi-Agent Reinforcement Learning

Multi-Agent DRL for Air-to-Ground Communication Planning in UAV-Enabled IoT Networks

Multi-Agent Deep Reinforcement Learning For Optimising Energy Efficiency of Fixed-Wing UAV Cellular Access Points

Multi-UAV Cooperative Search in Multi-Layered Aerial Computing Networks: A Multi-Agent Deep Reinforcement Learning Approach

Multi-Agent Reinforcement Learning for Offloading Cellular Communications with Cooperating UAVs

UAV Aided Search and Rescue Operation Using Reinforcement Learning

Optimizing cooperative cognitive search and rescue UAVs

Simultaneous Navigation and Radio Mapping for Cellular-Connected UAV With Deep Reinforcement Learning

Reinforcement Learning Placement Algorithm for Optimization of UAV Network in Wireless Communication

A Reinforcement Learning-Based Evolutionary Algorithm for the Unmanned Aerial Vehicles Maritime Search and Rescue Path Planning Problem Considering Multiple Rescue Centers

Multi-UAV Speed Control with Collision Avoidance and Handover-aware Cell Association: DRL with Action Branching

Enhanced Multi-UAV Path Planning in Complex Environments With Voronoi-Based Obstacle Modelling and Q-Learning

UAV Path Planning Optimization Strategy: Considerations of Urban Morphology, Microclimate, and Energy Efficiency Using Q-Learning Algorithm

Multi-UAV Path Learning for Age and Power Optimization in IoT With UAV Battery Recharge

Maritime Search and Rescue Leveraging Heterogeneous Units: A Multi-Agent Reinforcement Learning Approach.

Maritime Search and Rescue Based on Group Mobile Computing for Unmanned Aerial Vehicles and Unmanned Surface Vehicles

Simulation Analysis of Exploration Strategies and UAV Planning for Search and Rescue

Multi-Agent Reinforcement Learning Aided Resource Allocation with SARSA in UAV Networks

Optimizing UAV Trajectory for Emergency Response Operations under Real 3D Environments: Integrating Priority Levels and LoS Constraints