Abstract:Satellite-terrestrial integrated networks (STINs) are considered to be a new paradigm for the next generation of global communication because of its distinctive merits, such as wide coverage, high reliability, and flexibility. When the satellite associates with different base stations (BSs) and adopts different channels for communication, the utility of offloading data to BSs is different. In our work, we study how to jointly associate satellites with appropriate BSs and allocate channels to satellites. Our purpose is to maximize the utility of the data offloaded from satellites to BSs while considering the load balance of BSs. However, some satellites are often unable to connect to BSs because of their periodic flight characteristic, which makes the joint satellite-BS association and channel allocation more challenging. To solve the problem that satellites sometimes cannot connect to BSs, we abstract the communication model between satellites and BSs into a bipartite graph and add a virtual BS to ensure that all satellites can connect to at least one BS. Then, in the constructed joint optimization problem, we solve the assignment of satellites and channels simultaneously. Considering that the joint optimization problem is nonconvex, we use double deep Q-Network (DDQN) for achieving the optimal strategy of satellite association and channel allocation. Furthermore, the reward value in most state transition information generated by satellites is 0, which leads to the low learning efficiency of DDQN. Aiming at enhancing the learning efficiency of DDQN, the priority sampling-based DDQN (PSDDQN) algorithm is proposed. Experimental results demonstrate that PSDDQN gets better utility and achieves the load balance of BSs compared with other algorithms.

Satellite-Terrestrial Coordinated Multi-Satellite Beam Hopping Scheduling Based on Multi-Agent Deep Reinforcement Learning

Dynamic Beam Pattern and Bandwidth Allocation Based on Multi-Agent Deep Reinforcement Learning for Beam Hopping Satellite Systems

Deep Reinforcement Learning-Based Autonomous Mission Planning Method for High and Low Orbit Multiple Agile Earth Observing Satellites

Multi-Satellite Cooperative Load-Balancing Scheme Based on Dynamic Beam Coverage for LEO Beam Hopping Systems

Multi-Satellite Beam Hopping Based on Load Balancing and Interference Avoidance for NGSO Satellite Communication Systems

Demand-Aware Beam Hopping and Power Allocation for Load Balancing in Digital Twin empowered LEO Satellite Networks

Double-Timescale Multi-Agent Deep Reinforcement Learning for Flexible Payload in VHTS Systems

NGSO Satellites Beam Hopping Strategy Based on Load Balancing and Interference Avoidance for Coexistence with GSO Systems

Data-Driven Collaborative Scheduling Method for Multi-Satellite Data-Transmission

Multi - Agent Deep Deterministic Policy Gradient Based Satellite Spectrum/Code Resource Scheduling with Multi-constraint

Multi-Satellite Cooperative Networks: Joint Hybrid Beamforming and User Scheduling Design

A Hierarchical Resource Scheduling Method for Satellite Control System Based on Deep Reinforcement Learning

Multi-Agent Deep Reinforcement Learning Based Channel Allocation for Networked Satellite Telemetry System.

Deep Reinforcement Learning-Based Joint Satellite Scheduling and Resource Allocation in Satellite-Terrestrial Integrated Networks

Traffic Optimization in Satellites Communications: A Multi-agent Reinforcement Learning Approach

Resource Allocation for Beam-Hopping-Based Satellite Systems With Spectrum Sharing

Satellite Multi-beam Collaborative Scheduling in Satellite Aviation Communications

Joint Beam Scheduling and Power Optimization for Beam Hopping LEO Satellite Systems

Load-Aware Satellite Handover Strategy Based on Multi-Agent Reinforcement Learning

Multiobjective Optimization for Beam Hopping and Power Allocation in Dual Satellite Cooperative Transmission Networks

Energy-efficient traffic offloading for RSMA-based hybrid satellite terrestrial networks with deep reinforcement learning