Abstract:Abstract Under the background of the rapid development of network communication and satellite technology, multi-satellite online scheduling has brought many problems such as high labour costs, slow response speed, low task execution efficiency and low solution efficiency. How to effectively develop a real-time satellite scheduling scheme that can maximize the scheduling revenue is an important issue. In this paper, the multi-satellite online scheduling process is regarded as a Markov sequential decision process. When the task arrives, the scheduling decision can be made according to the current scheduling situation and the task information. Since the number and resources of satellites are fixed, but the information of the task itself such as the number of available time windows are unpredictable, this problem can be proved as an NP-Hard problem. There is no specific effective solution yet. As reinforcement learning algorithm has achieved good application results in decision-making optimization problems such as optimization decision, automatic control and production line scheduling, it is feasible to apply reinforcement learning method to multi-satellite online scheduling. Therefore, this paper studies the multi-satellite scheduling method based on reinforcement learning. This paper analyses the Markov property of the basic scheduling process of the multi-satellite online scheduling problem and establishes the corresponding Markov decision pro-cess. Considering the influence of the periodic schedule in the online scheduling on the scheduling strategy of the algorithm-decision network, an improved multi-satellite online scheduling model is proposed. Finally, the model is trained by the proximal policy optimization algorithm (PPO), and by comparing the simulation results with the existing multi-satellite online scheduling model, we verified the validity of our model.

Offline Learning-Based Multi-User Delay-Constrained Scheduling

Multi-Agent Reinforcement Learning Based Scheduling for Distributed PV-ESS Considering Incomplete Data Acquisition

Multi-User Delay-Constrained Scheduling With Deep Recurrent Reinforcement Learning

Effective Multi-User Delay-Constrained Scheduling with Deep Recurrent Reinforcement Learning

Online Multi-User Scheduling for XR Transmissions with Hard-Latency Constraint: Performance Analysis and Practical Design

Offline Reinforcement Learning for Learning to Dispatch for Job Shop Scheduling

Reinforcement Learning for Adaptive Resource Scheduling in Complex System Environments

An Advanced Reinforcement Learning Framework for Online Scheduling of Deferrable Workloads in Cloud Computing

Adaptive Multi-Core Real- Time Scheduling Based on Reinforcement Learning

Multi-Objective Order Scheduling via Reinforcement Learning

Learning to Schedule Online Tasks with Bandit Feedback

Learning-Augmented Scheduling

Online simulation task scheduling in cloud manufacturing with cross attention and deep reinforcement learning

Multi-satellite Online scheduling Method Based on Proximal Policy

Delay-Oriented Scheduling in 5G Downlink Wireless Networks Based on Reinforcement Learning With Partial Observations

Minimizing Mean Weighted Tardiness in Unrelated Parallel Machine Scheduling with Reinforcement Learning

Learning to Schedule Tasks with Deadline and Throughput Constraints.

A Two-stage Framework and Reinforcement Learning-based Optimization Algorithms for Complex Scheduling Problems

Developing Real-Time Scheduling Policy by Deep Reinforcement Learning

Reinforcement Learning Approach for Multi-Agent Flexible Scheduling Problems

A Scheduling Algorithm Based on Reinforcement Learning for Heterogeneous Environments.