Abstract:Unmanned aerial vehicles (UAVs) are increasingly deployed to provide wireless connectivity to static and mobile ground users in situations of increased network demand or points of failure in existing terrestrial cellular infrastructure. However, UAVs are energy-constrained and experience the challenge of interference from nearby UAV cells sharing the same frequency spectrum, thereby impacting the system's energy efficiency (EE). Recent approaches focus on optimising the system's EE by optimising the trajectory of UAVs serving only static ground users and neglecting mobile users. Several others neglect the impact of interference from nearby UAV cells, assuming an interference-free network environment. Despite growing research interest in decentralised control over centralised UAVs' control, direct collaboration among UAVs to improve coordination while optimising the systems' EE has not been adequately explored. To address this, we propose a direct collaborative communication-enabled multi-agent decentralised double deep Q-network (CMAD-DDQN) approach. The CMAD-DDQN is a collaborative algorithm that allows UAVs to explicitly share their telemetry via existing 3GPP guidelines by communicating with their nearest neighbours. This allows the agent-controlled UAVs to optimise their 3D flight trajectories by filling up knowledge gaps and converging to optimal policies. Simulation results show that the proposed approach outperforms existing baselines in terms of maximising the systems' EE without degrading coverage performance in the network. The CMAD-DDQN approach outperforms the MAD-DDQN that neglects direct collaboration among UAVs, the multi-agent deep deterministic policy gradient (MADDPG) and random policy approaches that consider a 2D UAV deployment design while neglecting interference from nearby UAV cells by about 15%, 65% and 85%, respectively.

Digital Twins-based Multi-agent Deep Reinforcement Learning for UAV-assisted Vehicle Edge Computing

Adaptive Digital Twin and Multiagent Deep Reinforcement Learning for Vehicular Edge Computing and Networks

Multi-Agent Deep Reinforcement Learning for Task Offloading in UAV-assisted Mobile Edge Computing

Multi-Agent DRL for Task Offloading and Resource Allocation in Multi-UAV Enabled IoT Edge Network

Digital Twin-Enabled Decision-Making Framework for Multi-UAV Mission Planning: A Multiagent Deep Reinforcement Learning Perspective

Digital Twin Assisted Task Offloading for Aerial Edge Computing and Networks

Digital Twin Assisted Task Assignment in Multi-UAV Systems: A Deep Reinforcement Learning Approach

Dense Multi-Agent Reinforcement Learning Aided Multi-UAV Information Coverage for Vehicular Networks

A multi-UAV assisted task offloading and path optimization for mobile edge computing via multi-agent deep reinforcement learning

Multi-UAV-assisted computation offloading in DT-based networks: A distributed deep reinforcement learning approach

Cooperative Multi-Agent Deep Reinforcement Learning Methods for UAV-aided Mobile Edge Computing Networks

Imperfect Digital Twin Assisted Low Cost Reinforcement Training for Multi-UAV Networks

Deep Reinforcement Learning based Path Planning for UAV-assisted Edge Computing Networks

Computation Offloading and Trajectory Control for UAV-Assisted Edge Computing Using Deep Reinforcement Learning

Deep Reinforcement Learning for Unmanned Aerial Vehicle-Assisted Vehicular Networks

Diffusion-based Reinforcement Learning for Dynamic UAV-assisted Vehicle Twins Migration in Vehicular Metaverses

Maximizing UAV Coverage in Maritime Wireless Networks: A Multiagent Reinforcement Learning Approach

Multi-Agent DRL for Air-to-Ground Communication Planning in UAV-Enabled IoT Networks

Communication-Enabled Deep Reinforcement Learning to Optimise Energy-Efficiency in UAV-Assisted Networks

Deep Reinforcement Learning for Scheduling and Offloading in UAV-Assisted Mobile Edge Networks