Abstract:Unmanned aerial vehicles (UAVs) are increasingly deployed to provide wireless connectivity to static and mobile ground users in situations of increased network demand or points of failure in existing terrestrial cellular infrastructure. However, UAVs are energy-constrained and experience the challenge of interference from nearby UAV cells sharing the same frequency spectrum, thereby impacting the system's energy efficiency (EE). Recent approaches focus on optimising the system's EE by optimising the trajectory of UAVs serving only static ground users and neglecting mobile users. Several others neglect the impact of interference from nearby UAV cells, assuming an interference-free network environment. Despite growing research interest in decentralised control over centralised UAVs' control, direct collaboration among UAVs to improve coordination while optimising the systems' EE has not been adequately explored. To address this, we propose a direct collaborative communication-enabled multi-agent decentralised double deep Q-network (CMAD-DDQN) approach. The CMAD-DDQN is a collaborative algorithm that allows UAVs to explicitly share their telemetry via existing 3GPP guidelines by communicating with their nearest neighbours. This allows the agent-controlled UAVs to optimise their 3D flight trajectories by filling up knowledge gaps and converging to optimal policies. Simulation results show that the proposed approach outperforms existing baselines in terms of maximising the systems' EE without degrading coverage performance in the network. The CMAD-DDQN approach outperforms the MAD-DDQN that neglects direct collaboration among UAVs, the multi-agent deep deterministic policy gradient (MADDPG) and random policy approaches that consider a 2D UAV deployment design while neglecting interference from nearby UAV cells by about 15%, 65% and 85%, respectively.

Reinforcement Learning Based Energy Efficient Robot Relay for Unmanned Aerial Vehicles Against Smart Jamming

Multi-Agent Reinforcement Learning Based UAV Swarm Communications Against Jamming

Distributed reinforcement learning based framework for energy-efficient UAV relay against jamming

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

Reinforcement-Learning-Based Relay Mobility and Power Allocation for Underwater Sensor Networks Against Jamming

Anti-Jamming UAV Relaying Based on Joint Geographic-Electromagnetic Domain Optimization

UAV Relay in VANETs Against Smart Jamming with Reinforcement Learning

UAV-Enabled Secure Communications by Multi-Agent Deep Reinforcement Learning

Reinforcement Learning-Based Anti-Jamming in Networked UAV Radar Systems

Jamming-Resilient Path Planning for Multiple UAVs via Deep Reinforcement Learning

Energy-Efficient Cooperative Relaying for Unmanned Aerial Vehicles.

Resilient Path Planning for UAVs in Data Collection under Adversarial Attacks

UAV Anti-Jamming Communications with Power and Mobility Control.

Energy Efficient Design in IRS-Assisted UAV Data Collection System under Malicious Jamming

Deep Reinforcement Learning for Joint Trajectory Planning, Transmission Scheduling, and Access Control in UAV-Assisted Wireless Sensor Networks

Blocklength Allocation and Power Control in UAV-Assisted URLLC System via Multi-agent Deep Reinforcement Learning

RIS-Assisted Jamming Rejection and Path Planning for UAV-Borne IoT Platform: A New Deep Reinforcement Learning Framework

Communication-Enabled Deep Reinforcement Learning to Optimise Energy-Efficiency in UAV-Assisted Networks

Multi-Agent Deep Reinforcement Learning for Secure UAV Communications

Multi-Agent Reinforcement Learning-Based Resource Allocation Scheme for UAV-Assisted Internet of Remote Things Systems

Reinforcement Learning Based Dynamic Power Control for UAV Mobility Management