Abstract:Without control center assistance, it is difficult that users take traffic loads of whole Ad Hoc networks into account to utilize wireless resources collaboratively. Considering influence of load differences between users on resource allocation can be learned in advance, we construct a framework of load-aware distributed resource allocation for multiple frequencies time division multiple access (MF-TDMA) wireless Ad Hoc networks based on multi-agent deep reinforcement learning (DRL) with centralized training and distributed execution. In this framework, each user is regarded as an agent and the wireless transmission resources are partitioned into time-frequency resource blocks (RBs). Each user reserves RBs by sending network manager operational packet (NMOP) in turn at rotating broadcast stage (RBS) and how many RBs each user reserves depends on the corresponding agent's decision. The main innovative point of our work is achieving a result of wireless resource allocation that balance load degree of all users on the whole network in a distributed manner. With defining a novel network utility parameterized by traffic loads (termed timely capacity), we first model the resource allocation as an optimization problem for timely capacity sum maximization. Using the optimization objective as common reward function to train neural networks of all agents in an offline simulation environment, we obtain the distributed algorithm based on multi-agent DRL. Experimental results demonstrate that with the neural networks trained by the designed reward and enough training, users can successfully learn to collaboratively improve network utility in distributed implementation.

Distributed Intelligence: A Verification for Multi-Agent DRL-Based Multibeam Satellite Resource Allocation

Multi-Agent DRL for Two-Timescale Bandwidth Allocation in Multi-Beam Satellite Networks

Multi-Agent Deep Reinforcement Learning-Based Flexible Satellite Payload for Mobile Terminals.

Computing Assistance from the Sky: Decentralized Computation Efficiency Optimization for Air-Ground Integrated MEC Networks

Multi-Agent DRL for User Association and Power Control in Terrestrial-Satellite Network.

Multi-Agent DRL for Resource Allocation and Cache Design in Terrestrial-Satellite Networks

An Online Distributed Satellite Cooperative Observation Scheduling Algorithm Based on Multiagent Deep Reinforcement Learning

DRL-Based Dynamic Resource Allocation for Multi-Beam Satellite Systems

Deep Reinforcement Learning Based Dynamic Channel Allocation Algorithm in Multibeam Satellite Systems

Dynamic Beam Pattern and Bandwidth Allocation Based on Multi-Agent Deep Reinforcement Learning for Beam Hopping Satellite Systems

Resource Allocation Using Deep Reinforcement Learning in GEO Multibeam Satellite System.

Dynamic Resource Allocation With Deep Reinforcement Learning in Multibeam Satellite Communication

Dynamic Channel Allocation for Satellite Internet of Things via Deep Reinforcement Learning

Deep Multiagent Reinforcement-Learning-Based Resource Allocation for Internet of Controllable Things

A Deep Reinforcement Learning-Based Framework for Dynamic Resource Allocation in Multibeam Satellite Systems.

A Novel Deep Reinforcement Learning Architecture for Dynamic Power and Bandwidth Allocation in Multibeam Satellites

Load-Aware Distributed Resource Allocation for MF-TDMA Ad Hoc Networks: A Multi-Agent DRL Approach.

Collaborative Deep Reinforcement Learning for Resource Optimization in Non-Terrestrial Networks

Deep Reinforcement Learning for Distributed Uncoordinated Cognitive Radios Resource Allocation