Abstract:When regarding the inherent uncertainty of differentiated services requirements as well as the non-uniform spatial distribution of capacity requests, it is essential to flexibility adjust resources of the satellite to satisfy the different conditions. How to match the system capacity demand with efficient utilization of beam is a brand-new challenge. The convention beam hopping methods ignores the intrinsic correlation between decisions, do not consider the long-term reward, and only achieve the optimal solution at the current time. Therefore, system complexity increases significantly as the increase of the demand for differentiated services or beam number. This paper investigates the optimal policy for beam hopping in DVB-S2X satellite with multiple purposes of assuring the fairness of each beam services, minimizing the delay of real-time services transmission, and maximizing the throughput of non-instant services transmission. Since wireless channel conditions, differentiated services arrival rates have stochastic properties, and the multi-beam satellite environment's dynamics are unknown, the model-free multi-objective deep reinforcement learning approach is used to learn the optimal policy through interactions with the situation. To solve the problem with action dimensional disaster, a novel multi-action selection method based on a Double-Loop Learning (DLL) is proposed. Moreover, the multi-dimensional state is reformulated and obtained by the deep neural network. Under realistic conditions achieving evaluation results demonstrate that the proposed method can pursue multiple objectives simultaneously, and it can also allocate resource intelligently adapting to the user requirements and channel conditions.

Deep Reinforcement Learning-Based Beam Hopping Algorithm in Multibeam Satellite Systems.

Multi-Satellite Beam Hopping and Power Allocation Using Deep Reinforcement Learning

Beam Hopping Scheduling Based on Deep Reinforcement Learning

Dynamic Beam Pattern and Bandwidth Allocation Based on Multi-Agent Deep Reinforcement Learning for Beam Hopping Satellite Systems

User-Level Dynamic Beam Hopping Design for LEO Satellite Networks Based on Deep Reinforcement Learning Assisted Enhanced Genetic Algorithm

Dynamic Beam Hopping for DVB-S2X Satellite: A Multi-Objective Deep Reinforcement Learning Approach

Dynamic Beam Hopping Method Based on Multi-Objective Deep Reinforcement Learning for Next Generation Satellite Broadband Systems

Towards Beam Hopping and Power Allocation in Multi-Beam Satellite Systems with Parameterized Reinforcement Learning

Dynamic Resource Allocation for Beam Hopping Satellites Communication System: an Exploration.

An Online Power Allocation Algorithm Based on Deep Reinforcement Learning in Multibeam Satellite Systems

Dynamic Beam Hopping for LEO Satellites with Differentiated Traffic Demands.

A Novel Deep Reinforcement Learning Architecture for Dynamic Power and Bandwidth Allocation in Multibeam Satellites

Dynamic Beam Hopping for DVB-S2X GEO Satellite: A DRL-Powered GA Approach

DRL-Based Dynamic Resource Allocation for Multi-Beam Satellite Systems

Sequential Dynamic Resource Allocation in Multi-Beam Satellite Systems: A Learning-Based Optimization Method

Traffic-Aware Resource Management of Beam Hopping in Satellite-Enabled Internet of Things

A Deep Reinforcement Learning-Based Framework for Dynamic Resource Allocation in Multibeam Satellite Systems.

Satellite-Terrestrial Coordinated Multi-Satellite Beam Hopping Scheduling Based on Multi-Agent Deep Reinforcement Learning

Dynamic Beam Hopping for Coverage Enhancement in Multi-Beam Satellite System Based on Determinantal Point Process Learning.

Deep Reinforcement Learning Based Dynamic Channel Allocation Algorithm in Multibeam Satellite Systems

Joint Beamforming Design for RIS-Assisted Integrated Satellite-HAP-Terrestrial Networks Using Deep Reinforcement Learning