Abstract:Unmanned aerial vehicles (UAVs) are increasingly deployed to provide wireless connectivity to static and mobile ground users in situations of increased network demand or points of failure in existing terrestrial cellular infrastructure. However, UAVs are energy-constrained and experience the challenge of interference from nearby UAV cells sharing the same frequency spectrum, thereby impacting the system's energy efficiency (EE). Recent approaches focus on optimising the system's EE by optimising the trajectory of UAVs serving only static ground users and neglecting mobile users. Several others neglect the impact of interference from nearby UAV cells, assuming an interference-free network environment. Despite growing research interest in decentralised control over centralised UAVs' control, direct collaboration among UAVs to improve coordination while optimising the systems' EE has not been adequately explored. To address this, we propose a direct collaborative communication-enabled multi-agent decentralised double deep Q-network (CMAD-DDQN) approach. The CMAD-DDQN is a collaborative algorithm that allows UAVs to explicitly share their telemetry via existing 3GPP guidelines by communicating with their nearest neighbours. This allows the agent-controlled UAVs to optimise their 3D flight trajectories by filling up knowledge gaps and converging to optimal policies. Simulation results show that the proposed approach outperforms existing baselines in terms of maximising the systems' EE without degrading coverage performance in the network. The CMAD-DDQN approach outperforms the MAD-DDQN that neglects direct collaboration among UAVs, the multi-agent deep deterministic policy gradient (MADDPG) and random policy approaches that consider a 2D UAV deployment design while neglecting interference from nearby UAV cells by about 15%, 65% and 85%, respectively.

Social-Aware Peer Selection for Energy Efficient D2D Communications in UAV-Assisted Networks: A Q-Learning Approach

Dynamic Social-Aware Peer Selection Scheme for Cooperative Device-to-Device Communications.

Robust Resource Allocation Algorithm for Energy-Harvesting-Based D2D Communication Underlaying UAV-Assisted Networks

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

Energy-Efficient Resource Allocation for Secure D2D Communications Underlaying UAV-Enabled Networks

Communication-Enabled Deep Reinforcement Learning to Optimise Energy-Efficiency in UAV-Assisted Networks

3D UAV Trajectory Design for Fair and Energy-Efficient Communication: A Deep Reinforcement Learning Technique

Dynamic Social-Aware Peer Selection for Cooperative Relay Management with D2D Communications

Social-Aware Peer Discovery for D2D Communications Underlaying Cellular Networks.

Energy-Efficient UAV Movement Control for Fair Communication Coverage: A Deep Reinforcement Learning Approach

Joint Deep Reinforcement Learning and Unsupervised Learning for Channel Selection and Power Control in D2D Networks

Learning-Enabled Radar-Assisted Predictive Beamforming for UAV-Aided Networks

Double Deep Q-Network Method for Energy Efficiency and Throughput in a UAV-Assisted Terrestrial Network

Energy Efficient D2D Communications Using Multiple UAV Relays

Learning-based Dynamic Connectivity Maintenance for UAV-assisted D2D Multicast Communication

Dynamic Trajectory and Power Control in Ultra-Dense UAV Networks: A Mean-Field Reinforcement Learning Approach

Opportunistic Utilization of Dynamic Multi-UAV in Device-to-Device Communication Networks

DQLEL: Deep Q-Learning for Energy-Optimized LoS/NLoS UWB Node Selection

A Hybrid Scheme Using TOPSIS and Q-Learning for Handover Decision Making in UAV Assisted Heterogeneous Network

Improved Q-learning based Multi-hop Routing for UAV-Assisted Communication

3D UAV Trajectory Design and Frequency Band Allocation for Energy-Efficient and Fair Communication: A Deep Reinforcement Learning Approach