Abstract:Due to the non-uniform geographic distribution and time-varying characteristics of the ground traffic request, how to make full use of the limited beam resources to serve users flexibly and efficiently is a brand-new challenge for beam hopping satellite systems. The conventional greedy-based beam hopping methods do not consider the long-term reward, which is difficult to deal with the time-varying traffic demand. Meanwhile, the heuristic algorithms such as genetic algorithm have a slow convergence time, which can not achieve real-time scheduling. Furthermore, existing methods based on deep reinforcement learning (DRL) only make decisions on beam patterns, lack of the freedom of bandwidth. This paper proposes a dynamic beam pattern and bandwidth allocation scheme based on DRL, which flexibly uses three degrees of freedom of time, space and frequency. Considering that the joint allocation of bandwidth and beam pattern will lead to an explosion of action space, a cooperative multi-agents deep reinforcement learning (MADRL) framework is presented in this paper, where each agent is only responsible for the illumination allocation or bandwidth allocation of one beam. The agents can learn to collaborate by sharing the same reward to achieve the common goal, which refers to maximize the throughput and minimize the delay fairness between cells. Simulation results demonstrate that the offline trained MADRL model can achieve real-time beam pattern and bandwidth allocation to match the non-uniform and time-varying traffic request. Furthermore, when the traffic demand increases, our model has a good generalization ability.

Demand Based Dynamic Bandwidth Allocation in Multibeam Satellites Using Machine Learning

DRL-Based Dynamic Resource Allocation for Multi-Beam Satellite Systems

Dynamic Frequency Assignment for Mobile Users in Multibeam Satellite Constellations

A Novel Deep Reinforcement Learning Architecture for Dynamic Power and Bandwidth Allocation in Multibeam Satellites

Flexible Payload Configuration for Satellites using Machine Learning

Research on Joint Resource Allocation for Multibeam Satellite Based on Metaheuristic Algorithms

+2cmFlexible Resource Management in High-throughput Satellite Communication Systems: A Two-stage Machine Learning Framework

Sequential Dynamic Resource Allocation in Multi-Beam Satellite Systems: A Learning-Based Optimization Method

Efficient Resource Allocation for Beam-Hopping-Based Multi-Satellite Communication Systems

Joint Beamforming Optimization for User-Centric Multi-Satellite System

Resource Allocation of Multibeam Communication Satellite Systems in Sparse Networks.

Intelligent Low Complexity Resource Allocation Method for Integrated Satellite-Terrestrial Systems

Supervised Learning Based Real-Time Adaptive Beamforming On-board Multibeam Satellites

Deep Reinforcement Learning Architecture for Continuous Power Allocation in High Throughput Satellites

Joint Beam Direction Control and Radio Resource Allocation in Dynamic Multi-beam LEO Satellite Networks

Demand-Aware Beam Hopping and Power Allocation for Load Balancing in Digital Twin empowered LEO Satellite Networks

Dynamic Resource Allocation With Deep Reinforcement Learning in Multibeam Satellite Communication

Dynamic Resource Allocation in LEO Satellite

Dynamic Beam Pattern and Bandwidth Allocation Based on Multi-Agent Deep Reinforcement Learning for Beam Hopping Satellite Systems