Abstract:Satellite networks have been regarded as a promising architecture for supporting the Internet of remote things (IoRT) due to their advantages of wide coverage and high communication capacity in remote areas, which further promotes the development of the satellites for IoRT networks (SIoRTNs). The effectiveness of multi-dimensional resource collaboration has significant impacts on the IoRT data downloading performance. However, the environment ' s dynamics, e.g., channel conditions and solar infeed process, are unknown in practical scenarios, which poses daunting challenges in making efficient utilization of multi-dimensional resources. Motivated by this fact, we model the joint resource scheduling and IoRT data scheduling problem with the aim of maximizing the amount of the IoRT data of the overall network by applying the model-free reinforcement learning framework. To overcome the limitations of traditional reinforcement learning algorithms, we propose several feature functions by investigating the natural attributes of the multi-dimensional resources of the SIoRTNs, and further exploit the concept of function approximation to approximate the expected downloaded IoRT data given the network state. Furthermore, we propose a state-action-reward-state-action (SARSA) based actor-critic reinforcement learning (SACRL) resource allocation strategy to achieve the optimal resource allocation and IoRT data scheduling with casual information at LEO satellites. Simulations validate the convergence property and the effectiveness of the proposed SACRL algorithm in terms of the amount of the downloaded IoRT data. Particularly, we investigate the impact of typical network parameters on network performance to further provide guidance for future SIoRTN system design.

Inter-Satellite Link Re-Planning Algorithm under Link Failures of LEO Satellite Constellations

Deep Reinforcement Learning-Based Autonomous Mission Planning Method for High and Low Orbit Multiple Agile Earth Observing Satellites

Traffic-aware Link Assignment in GEO Satellite Communication Networks

An Intelligent Routing Algorithm for LEO Satellites Based on Deep Reinforcement Learning

Reinforcement learning based dynamic distributed routing scheme for mega LEO satellite networks

Reinforcement Learning Based Intelligent Routing for Software Defined LEO Satellite Networks

SDDRL-SR: A High-Reliability Satellite Routing Algorithm Based on Deep Reinforcement Learning

A Routing Optimization Method for LEO Satellite Networks with Stochastic Link Failure

Inter-satellite Link Scheduling and Power Allocation Method for Satellite Networks

Optimization for Dynamic Laser Inter-Satellite Link Scheduling With Routing: A Multi-Agent Deep Reinforcement Learning Approach

Deep Reinforcement Learning-Based Joint Satellite Scheduling and Resource Allocation in Satellite-Terrestrial Integrated Networks

Multi-Commodity Flow Routing for Large-Scale LEO Satellite Networks Using Deep Reinforcement Learning

Real-Time Re-Planning for Satellite Formation Reconfiguration in Deep Space

Multi-Agent Deep Reinforcement Learning for Dynamic Laser Inter-Satellite Link Scheduling

Software Defined Routing Algorithm in LEO Satellite Networks

A DRL-Based Satellite Service Allocation Method in LEO Satellite Networks

Load-Aware Satellite Handover Strategy Based on Multi-Agent Reinforcement Learning

Dynamic Controller Placement for Software-Defined LEO Network Using Deep Reinforcement Learning

Improving Survivability of LEO Satellite Network with Guaranteed Based Approaches

Deep Reinforcement Learning for Congestion Control and Routing Optimization in LEO Satellite Networks

Machine Learning-Based Resource Allocation in Satellite Networks Supporting Internet of Remote Things