Abstract:Unmanned aerial vehicles (UAVs) are increasingly deployed to provide wireless connectivity to static and mobile ground users in situations of increased network demand or points of failure in existing terrestrial cellular infrastructure. However, UAVs are energy-constrained and experience the challenge of interference from nearby UAV cells sharing the same frequency spectrum, thereby impacting the system's energy efficiency (EE). Recent approaches focus on optimising the system's EE by optimising the trajectory of UAVs serving only static ground users and neglecting mobile users. Several others neglect the impact of interference from nearby UAV cells, assuming an interference-free network environment. Despite growing research interest in decentralised control over centralised UAVs' control, direct collaboration among UAVs to improve coordination while optimising the systems' EE has not been adequately explored. To address this, we propose a direct collaborative communication-enabled multi-agent decentralised double deep Q-network (CMAD-DDQN) approach. The CMAD-DDQN is a collaborative algorithm that allows UAVs to explicitly share their telemetry via existing 3GPP guidelines by communicating with their nearest neighbours. This allows the agent-controlled UAVs to optimise their 3D flight trajectories by filling up knowledge gaps and converging to optimal policies. Simulation results show that the proposed approach outperforms existing baselines in terms of maximising the systems' EE without degrading coverage performance in the network. The CMAD-DDQN approach outperforms the MAD-DDQN that neglects direct collaboration among UAVs, the multi-agent deep deterministic policy gradient (MADDPG) and random policy approaches that consider a 2D UAV deployment design while neglecting interference from nearby UAV cells by about 15%, 65% and 85%, respectively.

Deep Reinforcement Learning-based UAV Control for Optimized Energy Efficiency and Throughput in UAV-assisted Communication

Covert UAV Data Transmission Via Semantic Communication: A DRL-Driven Joint Position and Power Optimization Method

Joint 3D trajectory and phase shift optimization via deep reinforcement learning for RIS-assisted UAV communication systems

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

Deep Reinforcement Learning for Channel and Power Allocation in UAV-enabled IoT Systems

3D UAV Trajectory Design for Fair and Energy-Efficient Communication: A Deep Reinforcement Learning Technique

Deep Reinforcement Learning for Aerial Data Collection in Hybrid-Powered NOMA-IoT Networks

Blocklength Allocation and Power Control in UAV-Assisted URLLC System via Multi-agent Deep Reinforcement Learning

Deep Reinforcement Learning-enabled Dynamic UAV Deployment and Power Control in Multi-UAV Wireless Networks

A Novel Joint DRL-Based Utility Optimization for UAV Data Services

3D UAV Trajectory Design and Frequency Band Allocation for Energy-Efficient and Fair Communication: A Deep Reinforcement Learning Approach

A deep reinforcement learning framework and its implementation for UAV-aided covert communication

Deep Reinforcement Learning Based Trajectory Design and Resource Allocation for UAV-Assisted Communications

Communication-Enabled Deep Reinforcement Learning to Optimise Energy-Efficiency in UAV-Assisted Networks

Deep Reinforcement Learning Based 3D UAV Trajectory Design and Frequency Band Allocation.

Resource Allocation in UAV-Assisted Networks: A Clustering-Aided Reinforcement Learning Approach

Deep Reinforcement Learning for Unmanned Aerial Vehicle-Assisted Vehicular Networks

Reinforcement-Learning-Based Optimization on Energy Efficiency in UAV Networks for IoT

Autonomous Obstacle Avoidance and Communication Capacity Optimization for UAV-Assisted VLC Systems

Actor-Critic Deep Reinforcement Learning for Energy Minimization in UAV-Aided Networks

UAV-Assisted Wireless-Powered Secure Communications: Integration of Optimization and Deep Learning