Abstract:Satellite-terrestrial integrated networks (STINs) are considered to be a new paradigm for the next generation of global communication because of its distinctive merits, such as wide coverage, high reliability, and flexibility. When the satellite associates with different base stations (BSs) and adopts different channels for communication, the utility of offloading data to BSs is different. In our work, we study how to jointly associate satellites with appropriate BSs and allocate channels to satellites. Our purpose is to maximize the utility of the data offloaded from satellites to BSs while considering the load balance of BSs. However, some satellites are often unable to connect to BSs because of their periodic flight characteristic, which makes the joint satellite-BS association and channel allocation more challenging. To solve the problem that satellites sometimes cannot connect to BSs, we abstract the communication model between satellites and BSs into a bipartite graph and add a virtual BS to ensure that all satellites can connect to at least one BS. Then, in the constructed joint optimization problem, we solve the assignment of satellites and channels simultaneously. Considering that the joint optimization problem is nonconvex, we use double deep Q-Network (DDQN) for achieving the optimal strategy of satellite association and channel allocation. Furthermore, the reward value in most state transition information generated by satellites is 0, which leads to the low learning efficiency of DDQN. Aiming at enhancing the learning efficiency of DDQN, the priority sampling-based DDQN (PSDDQN) algorithm is proposed. Experimental results demonstrate that PSDDQN gets better utility and achieves the load balance of BSs compared with other algorithms.

Federated Deep Reinforcement Learning Assisting TT&C Mission Scheduling in Mega Satellite Networks

Deep Reinforcement Learning-Based Periodic Earth Observation Scheduling for Agile Satellite Constellation.

Deep Reinforcement Learning-Based Autonomous Mission Planning Method for High and Low Orbit Multiple Agile Earth Observing Satellites

Deep Reinforcement Learning-Based Satellite-Ground Links Scheduling for Mega Satellite Constellations

Collaborative Computing in Non-Terrestrial Networks: A Multi-Time-Scale Deep Reinforcement Learning Approach

Multi-Agent Deep Reinforcement Learning Based Channel Allocation for Networked Satellite Telemetry System.

Scheduling for On-Board Federated Learning with Satellite Clusters

Data-Driven Collaborative Scheduling Method for Multi-Satellite Data-Transmission

A Fast Approach to Satellite Range Rescheduling Using Deep Reinforcement Learning

Mission Planning for Distributed Multiple Agile Earth Observing Satellites by Attention-Based Deep Reinforcement Learning Method

Scheduling for Ground-Assisted Federated Learning in LEO Satellite Constellations

Dynamic Laser Inter-Satellite Link Scheduling Based on Federated Reinforcement Learning: An Asynchronous Hierarchical Architecture

Deep Reinforcement Learning-Based Joint Satellite Scheduling and Resource Allocation in Satellite-Terrestrial Integrated Networks

Reinforcement Learning for Cognitive Delay/Disruption Tolerant Network Node Management in an LEO-based Satellite Constellation

Deep Reinforcement Learning-Based Multipath Routing for LEO Megaconstellation Networks

Event-Triggered Deep Reinforcement Learning for Dynamic Task Scheduling in Multi-Satellite Resource Allocation

Multi-Commodity Flow Routing for Large-Scale LEO Satellite Networks Using Deep Reinforcement Learning

Learning to Construct a Solution for the Agile Satellite Scheduling Problem With Time-Dependent Transition Times

Collaborative Deep Reinforcement Learning for Resource Optimization in Non-Terrestrial Networks

Deep Reinforcement Learning for the Agile Earth Observation Satellite Scheduling Problem