Abstract:Due to the cooperative coverage characteristic of LEO satellites and non-uniform traffic demand of beam positions, allocating the limited beam and power resource to massive beam positions flexibly and effectively is a challenge in beam hopping LEO satellite communication system. The agents in existing beam hopping schemes, which rely on deep reinforcement learning, are limited to acquiring state information within the coverage area of LEO satellite. For this reason, we propose a cooperation multi-agent Value-Decomposition Networks with Dueling Double Deep Q-Learning Network (VDN-D3QN) framework to generate dynamic beam hopping pattern for assuring delay fairness and throughput among beam positions in LEO satellite communication system. The proposed VDN-D3QN dynamic beam hopping method is divided into training and test phase, where each agent is only responsible for the beam hopping pattern of one LEO satellite. During the train phase, the agents learn to cooperate with other agents to maximize the system throughput and minimize the delay fairness among beam positions by Dueling Double Deep Q-Learning Network. Then, the Value-Decomposition Networks is employed to learn the optimal policy in a centralized manner through interaction with the environment. In test phase, the trained agents are deployed to address the challenging problem of inter-satellite communication in a distributed manner, and one agent is deployed per LEO satellite. The trained agents can make decisions about the dynamic beam hopping pattern based on the available local state information in LEO satellite communication system. The evaluation results demonstrate that the proposed multi-agent VDN-D3QN algorithm can effectively handle the non-uniform traffic demand of multi-satellites simultaneously. Besides, the simulation results indicate that the proposed VDN-D3QN algorithm can allocate resource intelligently for adapting the requirements of beam positions and achieving better performance compared to the baselines.

Dynamic Beam Pattern Based on Cooperation Multi-Agent VDN-D3QN for LEO Satellite Communication System

Demand-Aware Beam Hopping and Power Allocation for Load Balancing in Digital Twin empowered LEO Satellite Networks

DDQN Based Beamwidth and Subcarrier Allocation Strategy for LEO Satellite Communication System with Multi-Beam Capability

Dynamic Beam Hopping Method Based on Multi-Objective Deep Reinforcement Learning for Next Generation Satellite Broadband Systems

Dynamic Beam Pattern and Bandwidth Allocation Based on Multi-Agent Deep Reinforcement Learning for Beam Hopping Satellite Systems

An Efficient Multi-Dimensional Resource Allocation Mechanism for Beam-Hopping in LEO Satellite Network

Dynamic Beam Hopping for LEO Satellites with Differentiated Traffic Demands.

Optimizing Beam Hopping in Multibeam NGSO Constellations with Multi-Agent Reinforcement Learning

Dynamic Beam Hopping for Coverage Enhancement in Multi-Beam Satellite System Based on Determinantal Point Process Learning.

Beam Hopping for Multi-Beam LEO Satellite Systems with Integrated Sensing and Communications

A Cooperative NOMA-Aided Multi-Dimensional Beam Hopping Method in Satellite Communication Systems.

Multi-Satellite Cooperative Load-Balancing Scheme Based on Dynamic Beam Coverage for LEO Beam Hopping Systems

Multi-Dimensional Resource Allocation Strategy for LEO Multi-Satellite Beam Hopping Systems

User Grouping-Based Beam Handover Scheme with Load-Balancing for LEO Satellite Networks

Satellite-Terrestrial Coordinated Multi-Satellite Beam Hopping Scheduling Based on Multi-Agent Deep Reinforcement Learning

A Muti-beam Placement Optimization Scheme in LEO Beam Hopping Satellite Systems.

Dynamic Beam Scheduling of Multi-NGSO Systems Based on Deep Reinforcement Learning

User-Level Dynamic Beam Hopping Design for LEO Satellite Networks Based on Deep Reinforcement Learning Assisted Enhanced Genetic Algorithm

System-Level Evaluation of Beam Hopping in NR-Based LEO Satellite Communication System

Multi-Agent Deep Reinforcement Learning-Based Flexible Satellite Payload for Mobile Terminals.

DRL-Based Dynamic Resource Allocation for Multi-Beam Satellite Systems