Abstract:There hardly exists a general solver that is efficient for scheduling problems due to their diversity and complexity. In this study, we develop a two-stage framework, in which reinforcement learning (RL) and traditional operations research (OR) algorithms are combined together to efficiently deal with complex scheduling problems. The scheduling problem is solved in two stages, including a finite Markov decision process (MDP) and a mixed-integer programming process, respectively. This offers a novel and general paradigm that combines RL with OR approaches to solving scheduling problems, which leverages the respective strengths of RL and OR: The MDP narrows down the search space of the original problem through an RL method, while the mixed-integer programming process is settled by an OR algorithm. These two stages are performed iteratively and interactively until the termination criterion has been met. Under this idea, two implementation versions of the combination methods of RL and OR are put forward. The agile Earth observation satellite scheduling problem is selected as an example to demonstrate the effectiveness of the proposed scheduling framework and methods. The convergence and generalization capability of the methods are verified by the performance of training scenarios, while the efficiency and accuracy are tested in 50 untrained scenarios. The results show that the proposed algorithms could stably and efficiently obtain satisfactory scheduling schemes for agile Earth observation satellite scheduling problems. In addition, it can be found that RL-based optimization algorithms have stronger scalability than non-learning algorithms. This work reveals the advantage of combining reinforcement learning methods with heuristic methods or mathematical programming methods for solving complex combinatorial optimization problems.

Learning to Construct a Solution for the Agile Satellite Scheduling Problem With Time-Dependent Transition Times

Deep Reinforcement Learning-Based Periodic Earth Observation Scheduling for Agile Satellite Constellation.

Deep Reinforcement Learning-Based Autonomous Mission Planning Method for High and Low Orbit Multiple Agile Earth Observing Satellites

Deep Reinforcement Learning for the Agile Earth Observation Satellite Scheduling Problem

A Heuristic Construction Neural Network Method for the Time-Dependent Agile Earth Observation Satellite Scheduling Problem

Deep Reinforcement Learning for Agile Satellite Scheduling Problem

Deep reinforcement learning and parameter transfer based approach for the multi-objective agile earth observation satellite scheduling problem

Two-Phase Neural Combinatorial Optimization with Reinforcement Learning for Agile Satellite Scheduling

Deep Reinforcement Learning Method for Satellite Range Scheduling Problem.

Tam-Ech: A Novel Method for Optimizing Multi-Agile Earth Observation Satellite Scheduling Problem Using Deep Q-Learning and Ensemble Heuristics

A Generic Markov Decision Process Model and Reinforcement Learning Method for Scheduling Agile Earth Observation Satellites

Deep Reinforcement Learning with Local Attention for Single Agile Optical Satellite Scheduling Problem

Mission Planning for Distributed Multiple Agile Earth Observing Satellites by Attention-Based Deep Reinforcement Learning Method

Deep Reinforcement Learning-Based Attention Decision Network for Agile Earth Observation Satellite Scheduling

A Fast Approach to Satellite Range Rescheduling Using Deep Reinforcement Learning

Task Scheduling under a Novel Framework for Data Relay Satellite Network Via Deep Reinforcement Learning

DRL-Based Dynamic Destroy Approaches for Agile-Satellite Mission Planning

An Integrated Scheduling Algorithm of Satellite Observation and Data Download Based on Multi-Agent Deep Reinforcement Learning

A Two-stage Framework and Reinforcement Learning-based Optimization Algorithms for Complex Scheduling Problems

Revising the Observation Satellite Scheduling Problem Based on Deep Reinforcement Learning

Event-Triggered Deep Reinforcement Learning for Dynamic Task Scheduling in Multisatellite Resource Allocation.