Abstract:Satellite-terrestrial integrated networks (STINs) are considered to be a new paradigm for the next generation of global communication because of its distinctive merits, such as wide coverage, high reliability, and flexibility. When the satellite associates with different base stations (BSs) and adopts different channels for communication, the utility of offloading data to BSs is different. In our work, we study how to jointly associate satellites with appropriate BSs and allocate channels to satellites. Our purpose is to maximize the utility of the data offloaded from satellites to BSs while considering the load balance of BSs. However, some satellites are often unable to connect to BSs because of their periodic flight characteristic, which makes the joint satellite-BS association and channel allocation more challenging. To solve the problem that satellites sometimes cannot connect to BSs, we abstract the communication model between satellites and BSs into a bipartite graph and add a virtual BS to ensure that all satellites can connect to at least one BS. Then, in the constructed joint optimization problem, we solve the assignment of satellites and channels simultaneously. Considering that the joint optimization problem is nonconvex, we use double deep Q-Network (DDQN) for achieving the optimal strategy of satellite association and channel allocation. Furthermore, the reward value in most state transition information generated by satellites is 0, which leads to the low learning efficiency of DDQN. Aiming at enhancing the learning efficiency of DDQN, the priority sampling-based DDQN (PSDDQN) algorithm is proposed. Experimental results demonstrate that PSDDQN gets better utility and achieves the load balance of BSs compared with other algorithms.

Deep Reinforcement Learning-Based Satellite-Ground Links Scheduling for Mega Satellite Constellations

Deep Reinforcement Learning-Based Periodic Earth Observation Scheduling for Agile Satellite Constellation.

Deep Reinforcement Learning-Based Autonomous Mission Planning Method for High and Low Orbit Multiple Agile Earth Observing Satellites

Event-Triggered Deep Reinforcement Learning for Dynamic Task Scheduling in Multi-Satellite Resource Allocation

Deep Reinforcement Learning-Based Joint Satellite Scheduling and Resource Allocation in Satellite-Terrestrial Integrated Networks

A Fast Approach to Satellite Range Rescheduling Using Deep Reinforcement Learning

A robust routing strategy based on deep reinforcement learning for mega satellite constellations

Data-Driven Collaborative Scheduling Method for Multi-Satellite Data-Transmission

Dynamic Resource Allocation With Deep Reinforcement Learning in Multibeam Satellite Communication

Multi-Commodity Flow Routing for Large-Scale LEO Satellite Networks Using Deep Reinforcement Learning

Deep Reinforcement Learning-Based Multipath Routing for LEO Megaconstellation Networks

Graph Neural Network and Reinforcement Learning Based Routing for Mega LEO Satellite Constellations

A Hierarchical Resource Scheduling Method for Satellite Control System Based on Deep Reinforcement Learning

BeiDou Short-Message Satellite Resource Allocation Algorithm Based on Deep Reinforcement Learning

Reinforcement learning based dynamic distributed routing scheme for mega LEO satellite networks

SDDRL-SR: A High-Reliability Satellite Routing Algorithm Based on Deep Reinforcement Learning

Dynamic Laser Inter-Satellite Link Scheduling Based on Federated Reinforcement Learning: An Asynchronous Hierarchical Architecture

Mission Planning for Distributed Multiple Agile Earth Observing Satellites by Attention-Based Deep Reinforcement Learning Method

Deep Reinforcement Learning for Congestion Control and Routing Optimization in LEO Satellite Networks

Deep Reinforcement Learning Architecture for Continuous Power Allocation in High Throughput Satellites