Abstract:Unmanned aerial vehicles (UAVs) are increasingly deployed to provide wireless connectivity to static and mobile ground users in situations of increased network demand or points of failure in existing terrestrial cellular infrastructure. However, UAVs are energy-constrained and experience the challenge of interference from nearby UAV cells sharing the same frequency spectrum, thereby impacting the system's energy efficiency (EE). Recent approaches focus on optimising the system's EE by optimising the trajectory of UAVs serving only static ground users and neglecting mobile users. Several others neglect the impact of interference from nearby UAV cells, assuming an interference-free network environment. Despite growing research interest in decentralised control over centralised UAVs' control, direct collaboration among UAVs to improve coordination while optimising the systems' EE has not been adequately explored. To address this, we propose a direct collaborative communication-enabled multi-agent decentralised double deep Q-network (CMAD-DDQN) approach. The CMAD-DDQN is a collaborative algorithm that allows UAVs to explicitly share their telemetry via existing 3GPP guidelines by communicating with their nearest neighbours. This allows the agent-controlled UAVs to optimise their 3D flight trajectories by filling up knowledge gaps and converging to optimal policies. Simulation results show that the proposed approach outperforms existing baselines in terms of maximising the systems' EE without degrading coverage performance in the network. The CMAD-DDQN approach outperforms the MAD-DDQN that neglects direct collaboration among UAVs, the multi-agent deep deterministic policy gradient (MADDPG) and random policy approaches that consider a 2D UAV deployment design while neglecting interference from nearby UAV cells by about 15%, 65% and 85%, respectively.

Energy-efficient UAV Trajectory Design for Backscatter Communication: A Deep Reinforcement Learning Approach

3D UAV Trajectory Design and Frequency Band Allocation for Energy-Efficient and Fair Communication: A Deep Reinforcement Learning Approach

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

3D UAV Trajectory Design for Fair and Energy-Efficient Communication: A Deep Reinforcement Learning Technique

Deep Reinforcement Learning Based 3D UAV Trajectory Design and Frequency Band Allocation.

Joint Trajectory and Passive Beamforming Design for Intelligent Reflecting Surface-Aided UAV Communications: A Deep Reinforcement Learning Approach

Movable Antenna-Equipped UAV for Data Collection in Backscatter Sensor Networks: A Deep Reinforcement Learning-based Approach

Radio Resource Management for Cellular-Connected UAV: A Learning Approach

UAV Trajectory Optimization for Large-Scale and Low-Power Data Collection: An Attention-Reinforced Learning Scheme

Joint 3D trajectory and phase shift optimization via deep reinforcement learning for RIS-assisted UAV communication systems

Intelligent flying-beamformer for hybrid mmWave systems: A deep reinforcement learning approach

Cellular UAV-to-Device Communications: Trajectory Design and Mode Selection by Multi-agent Deep Reinforcement Learning

Communication-Enabled Deep Reinforcement Learning to Optimise Energy-Efficiency in UAV-Assisted Networks

UAV Trajectory and Energy Efficiency Optimization in RIS-Assisted Multi-User Air-to-Ground Communications Networks

Computation Offloading and Trajectory Control for UAV-Assisted Edge Computing Using Deep Reinforcement Learning

Three-dimensional deep reinforcement learning for trajectory and resource optimization in UAV communication systems

Hierarchical Deep Reinforcement Learning for Backscattering Data Collection With Multiple UAVs

3D-Trajectory and Phase-Shift Design for RIS-Assisted UAV Systems Using Deep Reinforcement Learning

On Designing Multi-UAV aided Wireless Powered Dynamic Communication via Hierarchical Deep Reinforcement Learning

Joint optimization of resource allocation and trajectory based on user trajectory for UAV-assisted backscatter communication system

Multi-Agent Deep Reinforcement Learning For Optimising Energy Efficiency of Fixed-Wing UAV Cellular Access Points