Abstract:When regarding the inherent uncertainty of differentiated services requirements as well as the non-uniform spatial distribution of capacity requests, it is essential to flexibility adjust resources of the satellite to satisfy the different conditions. How to match the system capacity demand with efficient utilization of beam is a brand-new challenge. The convention beam hopping methods ignores the intrinsic correlation between decisions, do not consider the long-term reward, and only achieve the optimal solution at the current time. Therefore, system complexity increases significantly as the increase of the demand for differentiated services or beam number. This paper investigates the optimal policy for beam hopping in DVB-S2X satellite with multiple purposes of assuring the fairness of each beam services, minimizing the delay of real-time services transmission, and maximizing the throughput of non-instant services transmission. Since wireless channel conditions, differentiated services arrival rates have stochastic properties, and the multi-beam satellite environment's dynamics are unknown, the model-free multi-objective deep reinforcement learning approach is used to learn the optimal policy through interactions with the situation. To solve the problem with action dimensional disaster, a novel multi-action selection method based on a Double-Loop Learning (DLL) is proposed. Moreover, the multi-dimensional state is reformulated and obtained by the deep neural network. Under realistic conditions achieving evaluation results demonstrate that the proposed method can pursue multiple objectives simultaneously, and it can also allocate resource intelligently adapting to the user requirements and channel conditions.

Deep Reinforcement Learning for Dynamic Bandwidth Allocation in Multi-Beam Satellite Systems

Towards Beam Hopping and Power Allocation in Multi-Beam Satellite Systems with Parameterized Reinforcement Learning

Sequential Dynamic Resource Allocation in Multi-Beam Satellite Systems: A Learning-Based Optimization Method

A Novel Deep Reinforcement Learning Architecture for Dynamic Power and Bandwidth Allocation in Multibeam Satellites

A Deep Reinforcement Learning-Based Framework for Dynamic Resource Allocation in Multibeam Satellite Systems.

Dynamic Beam Pattern and Bandwidth Allocation Based on Multi-Agent Deep Reinforcement Learning for Beam Hopping Satellite Systems

Resource Allocation Using Deep Reinforcement Learning in GEO Multibeam Satellite System.

Deep Reinforcement Learning Based Dynamic Channel Allocation Algorithm in Multibeam Satellite Systems

Multi-Agent DRL for Two-Timescale Bandwidth Allocation in Multi-Beam Satellite Networks

Dynamic Beam Hopping Method Based on Multi-Objective Deep Reinforcement Learning for Next Generation Satellite Broadband Systems

DRL-Based Dynamic Resource Allocation for Multi-Beam Satellite Systems

An Online Power Allocation Algorithm Based on Deep Reinforcement Learning in Multibeam Satellite Systems

Dynamic Resource Allocation for Beam Hopping Satellites Communication System: an Exploration.

DDQN Based Beamwidth and Subcarrier Allocation Strategy for LEO Satellite Communication System with Multi-Beam Capability

Deep Reinforcement Learning-Based Dynamic Multi-Beam Power Allocation for GEO-LEO Co-Existing Satellites

Optimum Power Allocation Based on Traffic Matching Service for Multi-beam Satellite System

Collaborative Long-Short Term Bandwidth Allocation for Satellite-Terrestrial Networks

A DRL Resource Allocation for Downlink NOMA Multi-beam Satellite Communications.

Dynamic Bandwidth Allocation for Multi-Qos Guarantee Based on Bee Colony Optimization

Deep Reinforcement Learning-Based Beam Hopping Algorithm in Multibeam Satellite Systems.

Multi-Agent Deep Reinforcement Learning-Based Flexible Satellite Payload for Mobile Terminals.