Abstract:Directional unmanned aerial vehicle (UAV) ad hoc networks (DUANETs) are widely applied due to their high flexibility, strong anti-interference capability, and high transmission rates. However, within directional networks, complex mutual interference persists, necessitating scheduling of the time slot, power, and main lobe direction for all links to improve the transmission performance of DUANETs. To ensure transmission fairness and the total count of transmitted data packets for the DUANET under dynamic data transmission demands, a scheduling algorithm for the time slot, power, and main lobe direction based on multi-agent deep reinforcement learning (MADRL) is proposed. Specifically, modeling is performed with the links as the core, optimizing the time slot, power, and main lobe direction variables for the fairness-weighted count of transmitted data packets. A decentralized partially observable Markov decision process (Dec-POMDP) is constructed for the problem. To process the observation in Dec-POMDP, an attention mechanism-based observation processing method is proposed to extract observation features of UAVs and their neighbors within the main lobe range, enhancing algorithm performance. The proposed Dec-POMDP and MADRL algorithms enable distributed autonomous decision-making for the resource scheduling of time slots, power, and main lobe directions. Finally, the simulation and analysis are primarily focused on the performance of the proposed algorithm and existing algorithms across varying data packet generation rates, different main lobe gains, and varying main lobe widths. The simulation results show that the proposed attention mechanism-based MADRL algorithm enhances the performance of the MADRL algorithm by 22.17%. The algorithm with the main lobe direction scheduling improves performance by 67.06% compared to the algorithm without the main lobe direction scheduling.

Multi-Agent Deep Reinforcement Learning for Dynamic Laser Inter-Satellite Link Scheduling

Optimization for Dynamic Laser Inter-Satellite Link Scheduling With Routing: A Multi-Agent Deep Reinforcement Learning Approach

Dynamic Laser Inter-Satellite Link Scheduling Based on Federated Reinforcement Learning: An Asynchronous Hierarchical Architecture

Deep Reinforcement Learning-Based Autonomous Mission Planning Method for High and Low Orbit Multiple Agile Earth Observing Satellites

Deep Reinforcement Learning-Based Periodic Earth Observation Scheduling for Agile Satellite Constellation.

Dynamic Beam Pattern and Bandwidth Allocation Based on Multi-Agent Deep Reinforcement Learning for Beam Hopping Satellite Systems

Distributed Satellite Cluster Laser Networking Algorithm with Double-Layer Markov DRL Architecture

Satellite-Terrestrial Coordinated Multi-Satellite Beam Hopping Scheduling Based on Multi-Agent Deep Reinforcement Learning

On-Demand Routing in LEO Mega-Constellations with Dynamic Laser Inter-Satellite Links

DRL-Based Dynamic Resource Allocation for Multi-Beam Satellite Systems

Joint Resource Scheduling of the Time Slot, Power, and Main Lobe Direction in Directional UAV Ad Hoc Networks: A Multi-Agent Deep Reinforcement Learning Approach

Event-Triggered Deep Reinforcement Learning for Dynamic Task Scheduling in Multi-Satellite Resource Allocation

Dynamic Routing for Integrated Satellite-Terrestrial Networks: A Constrained Multi-Agent Reinforcement Learning Approach

Collaborative Intelligent Reflecting Surface Networks with Multi-Agent Reinforcement Learning

A Policy Based Deep Reinforcement Learning for Task Offloading and Resource Allocation in Satellite Terrestrial Integrated Internet of Things

A Fast Approach to Satellite Range Rescheduling Using Deep Reinforcement Learning

Joint Task Offloading and Resource Allocation for Intelligent Reflecting Surface-Aided Integrated Sensing and Communication Systems Using Deep Reinforcement Learning Algorithm

Double-Timescale Multi-Agent Deep Reinforcement Learning for Flexible Payload in VHTS Systems

Deep Reinforcement Learning-Based Satellite-Ground Links Scheduling for Mega Satellite Constellations

Multi - Agent Deep Deterministic Policy Gradient Based Satellite Spectrum/Code Resource Scheduling with Multi-constraint

Joint Sensing and Communication Optimization in Target-Mounted STARS-Assisted Vehicular Networks: A MADRL Approach