Abstract:Satellite-terrestrial integrated networks (STINs) are considered to be a new paradigm for the next generation of global communication because of its distinctive merits, such as wide coverage, high reliability, and flexibility. When the satellite associates with different base stations (BSs) and adopts different channels for communication, the utility of offloading data to BSs is different. In our work, we study how to jointly associate satellites with appropriate BSs and allocate channels to satellites. Our purpose is to maximize the utility of the data offloaded from satellites to BSs while considering the load balance of BSs. However, some satellites are often unable to connect to BSs because of their periodic flight characteristic, which makes the joint satellite-BS association and channel allocation more challenging. To solve the problem that satellites sometimes cannot connect to BSs, we abstract the communication model between satellites and BSs into a bipartite graph and add a virtual BS to ensure that all satellites can connect to at least one BS. Then, in the constructed joint optimization problem, we solve the assignment of satellites and channels simultaneously. Considering that the joint optimization problem is nonconvex, we use double deep Q-Network (DDQN) for achieving the optimal strategy of satellite association and channel allocation. Furthermore, the reward value in most state transition information generated by satellites is 0, which leads to the low learning efficiency of DDQN. Aiming at enhancing the learning efficiency of DDQN, the priority sampling-based DDQN (PSDDQN) algorithm is proposed. Experimental results demonstrate that PSDDQN gets better utility and achieves the load balance of BSs compared with other algorithms.

Reconfigurable Topology Design with Deep Reinforcement Learning in Satellite Optical Network

A Topology Design Method for Satellite Networks Based on Deep Reinforcement Learning

Reconfigurable Network Topology Based on Deep Reinforcement Learning in Software-Defined Data-Center Networks

Study on Topology Reconfiguration of Optical Interconnection Network in Data Center

Automatically Reconfigurable Optical Network for HPC System Based on Deep Reinforcement Learning

SDDRL-SR: A High-Reliability Satellite Routing Algorithm Based on Deep Reinforcement Learning

An Intelligent Routing Algorithm for LEO Satellites Based on Deep Reinforcement Learning

GRouting: Dynamic Routing for LEO Satellite Networks with Graph-based Deep Reinforcement Learning

Reinforcement learning based dynamic distributed routing scheme for mega LEO satellite networks

Deep Reinforcement Learning-Based Joint Satellite Scheduling and Resource Allocation in Satellite-Terrestrial Integrated Networks

Multi-Commodity Flow Routing for Large-Scale LEO Satellite Networks Using Deep Reinforcement Learning

Network Topology Optimization via Deep Reinforcement Learning

Enabling High-Throughput Routing for LEO Satellite Broadband Networks: A Flow-Centric Deep Reinforcement Learning Approach

Deep Reinforcement Learning for Congestion Control and Routing Optimization in LEO Satellite Networks

Dynamic Controller Placement for Software-Defined LEO Network Using Deep Reinforcement Learning

Reinforcement Learning Based Intelligent Routing for Software Defined LEO Satellite Networks

Optimizing Deep-Space DTN Congestion Control Via Deep Reinforcement Learning

Reconfiguring Multicast Sessions in EONs Adaptively with Deep Reinforcement Learning: (Invited Paper)

Satellite Attitude Control with Deep Reinforcement Learning

A Novel Topology Design Method For Multi-Layered Optical Satellite Networks

Collaborative Deep Reinforcement Learning for Resource Optimization in Non-Terrestrial Networks