Abstract:Due to the cooperative coverage characteristic of LEO satellites and non-uniform traffic demand of beam positions, allocating the limited beam and power resource to massive beam positions flexibly and effectively is a challenge in beam hopping LEO satellite communication system. The agents in existing beam hopping schemes, which rely on deep reinforcement learning, are limited to acquiring state information within the coverage area of LEO satellite. For this reason, we propose a cooperation multi-agent Value-Decomposition Networks with Dueling Double Deep Q-Learning Network (VDN-D3QN) framework to generate dynamic beam hopping pattern for assuring delay fairness and throughput among beam positions in LEO satellite communication system. The proposed VDN-D3QN dynamic beam hopping method is divided into training and test phase, where each agent is only responsible for the beam hopping pattern of one LEO satellite. During the train phase, the agents learn to cooperate with other agents to maximize the system throughput and minimize the delay fairness among beam positions by Dueling Double Deep Q-Learning Network. Then, the Value-Decomposition Networks is employed to learn the optimal policy in a centralized manner through interaction with the environment. In test phase, the trained agents are deployed to address the challenging problem of inter-satellite communication in a distributed manner, and one agent is deployed per LEO satellite. The trained agents can make decisions about the dynamic beam hopping pattern based on the available local state information in LEO satellite communication system. The evaluation results demonstrate that the proposed multi-agent VDN-D3QN algorithm can effectively handle the non-uniform traffic demand of multi-satellites simultaneously. Besides, the simulation results indicate that the proposed VDN-D3QN algorithm can allocate resource intelligently for adapting the requirements of beam positions and achieving better performance compared to the baselines.

Multi-Agent Deep Reinforcement Learning-Based Flexible Satellite Payload for Mobile Terminals.

Dynamic Beam Pattern and Bandwidth Allocation Based on Multi-Agent Deep Reinforcement Learning for Beam Hopping Satellite Systems

Multi-Agent Deep Reinforcement Learning Based Channel Allocation for Networked Satellite Telemetry System.

Satellite-Terrestrial Coordinated Multi-Satellite Beam Hopping Scheduling Based on Multi-Agent Deep Reinforcement Learning

Dynamic Beam Hopping Method Based on Multi-Objective Deep Reinforcement Learning for Next Generation Satellite Broadband Systems

Double-Timescale Multi-Agent Deep Reinforcement Learning for Flexible Payload in VHTS Systems

Distributed Intelligence: A Verification for Multi-Agent DRL-Based Multibeam Satellite Resource Allocation

Optimizing Beam Hopping in Multibeam NGSO Constellations with Multi-Agent Reinforcement Learning

Efficient Resource Allocation for Multi-Beam Satellite-Terrestrial Vehicular Networks: A Multi-Agent Actor-Critic Method With Attention Mechanism

Load-Aware Satellite Handover Strategy Based on Multi-Agent Reinforcement Learning

An Online Distributed Satellite Cooperative Observation Scheduling Algorithm Based on Multiagent Deep Reinforcement Learning

Flexible Robust Beamforming for Multibeam Satellite Downlink using Reinforcement Learning

Towards Beam Hopping and Power Allocation in Multi-Beam Satellite Systems with Parameterized Reinforcement Learning

Deep Reinforcement Learning for Dynamic Bandwidth Allocation in Multi-Beam Satellite Systems

Multi-Agent DRL for Two-Timescale Bandwidth Allocation in Multi-Beam Satellite Networks

Collaborate Q-learning Aided Load Balance in Satellites Communications

DRL-Based Dynamic Resource Allocation for Multi-Beam Satellite Systems

Multi-Agent DRL for Resource Allocation and Cache Design in Terrestrial-Satellite Networks

Multi-Agent DRL for User Association and Power Control in Terrestrial-Satellite Network.

Dynamic Beam Pattern Based on Cooperation Multi-Agent VDN-D3QN for LEO Satellite Communication System

A Novel Deep Reinforcement Learning Architecture for Dynamic Power and Bandwidth Allocation in Multibeam Satellites