Abstract:Unmanned aerial vehicles (UAVs) are increasingly deployed to provide wireless connectivity to static and mobile ground users in situations of increased network demand or points of failure in existing terrestrial cellular infrastructure. However, UAVs are energy-constrained and experience the challenge of interference from nearby UAV cells sharing the same frequency spectrum, thereby impacting the system's energy efficiency (EE). Recent approaches focus on optimising the system's EE by optimising the trajectory of UAVs serving only static ground users and neglecting mobile users. Several others neglect the impact of interference from nearby UAV cells, assuming an interference-free network environment. Despite growing research interest in decentralised control over centralised UAVs' control, direct collaboration among UAVs to improve coordination while optimising the systems' EE has not been adequately explored. To address this, we propose a direct collaborative communication-enabled multi-agent decentralised double deep Q-network (CMAD-DDQN) approach. The CMAD-DDQN is a collaborative algorithm that allows UAVs to explicitly share their telemetry via existing 3GPP guidelines by communicating with their nearest neighbours. This allows the agent-controlled UAVs to optimise their 3D flight trajectories by filling up knowledge gaps and converging to optimal policies. Simulation results show that the proposed approach outperforms existing baselines in terms of maximising the systems' EE without degrading coverage performance in the network. The CMAD-DDQN approach outperforms the MAD-DDQN that neglects direct collaboration among UAVs, the multi-agent deep deterministic policy gradient (MADDPG) and random policy approaches that consider a 2D UAV deployment design while neglecting interference from nearby UAV cells by about 15%, 65% and 85%, respectively.

Energy-Efficient Resource Management for Multi-UAV NOMA Networks Based on Deep Reinforcement Learning

Deep reinforcement learning in NOMA-assisted UAV networks for path selection and resource offloading

Matching-Based Resource Allocation and Distributed Power Control Using Mean Field Game in the NOMA-Based UAV Networks.

DRL-based Joint Optimization for Energy Efficiency Maximization in DAV-NOMA Networks.

Multi-Agent Reinforcement Learning in NOMA-aided UAV Networks for Cellular Offloading

Deep Reinforcement Learning Assisted UAV Trajectory and Resource Optimization for NOMA Networks

Height Optimization and Resource Allocation for NOMA Enhanced UAV-Aided Relay Networks

Energy-Efficient Resource Allocation in a Multi-UAV-Aided NOMA Network

Energy-Efficient Joint Scheduling and Resource Management for UAV-Enabled Multicell Networks.

Resource Allocation in UAV-Assisted Networks: A Clustering-Aided Reinforcement Learning Approach

Deep Reinforcement Learning Based Resource Allocation in Multi-UAV-Aided MEC Networks.

Resource Allocation for Multi-Cluster NOMA-UAV Networks

Resource Allocation for the Backhaul of NOMA-Based Cellular UAV Network.

Radio Resource Management for Cellular-Connected UAV: A Learning Approach

Constrained Deep Reinforcement Learning for Energy Sustainable Multi-UAV based Random Access IoT Networks with NOMA

Energy-efficient Design for Mmwave-Enabled NOMA-UAV Networks

Energy-efficient UAV communication: A NOMA scheme with resource allocation and trajectory optimization

Deep Reinforcement Learning for Aerial Data Collection in Hybrid-Powered NOMA-IoT Networks

Communication-Enabled Deep Reinforcement Learning to Optimise Energy-Efficiency in UAV-Assisted Networks

Deep Reinforcement Learning-Based Resource Management for UAV-Assisted Mobile Edge Computing Against Jamming

Energy-Efficient Resource Allocation for Dual-NOMA-UAV Assisted Internet of Things