Abstract:There hardly exists a general solver that is efficient for scheduling problems due to their diversity and complexity. In this study, we develop a two-stage framework, in which reinforcement learning (RL) and traditional operations research (OR) algorithms are combined together to efficiently deal with complex scheduling problems. The scheduling problem is solved in two stages, including a finite Markov decision process (MDP) and a mixed-integer programming process, respectively. This offers a novel and general paradigm that combines RL with OR approaches to solving scheduling problems, which leverages the respective strengths of RL and OR: The MDP narrows down the search space of the original problem through an RL method, while the mixed-integer programming process is settled by an OR algorithm. These two stages are performed iteratively and interactively until the termination criterion has been met. Under this idea, two implementation versions of the combination methods of RL and OR are put forward. The agile Earth observation satellite scheduling problem is selected as an example to demonstrate the effectiveness of the proposed scheduling framework and methods. The convergence and generalization capability of the methods are verified by the performance of training scenarios, while the efficiency and accuracy are tested in 50 untrained scenarios. The results show that the proposed algorithms could stably and efficiently obtain satisfactory scheduling schemes for agile Earth observation satellite scheduling problems. In addition, it can be found that RL-based optimization algorithms have stronger scalability than non-learning algorithms. This work reveals the advantage of combining reinforcement learning methods with heuristic methods or mathematical programming methods for solving complex combinatorial optimization problems.

Efficient Rolling Horizon Algorithms for Resources-Constrained Scheduling Problems with Buffers

AN ALGORITHM FOR THE JOB SCHEDULE PROBLEM ON THE MODEL OF THE COMMON RESOURCES CAPTURED IN SHORT TIME

Research on Optimal Two Element Exchange Algorithm for Large Scale Cloud Computing Server Scheduling Problem

Some Results on Resource Constrained Scheduling

An Efficient Fault-Tolerant Scheduling Algorithm for Periodic Real-Time Tasks in Heterogeneous Platforms

An Improved Genetic Algorithm with Local Search for Dynamic Job Shop Scheduling Problem.

Research on Any Window Constraint Scheduling in Weakly Hard Real-Time System

Rolling Scheduling Method considering Shiftable Demand Respond Resources

Minimizing the Communication Overhead of Iterative Scheduling Algorithms for Input-Queued Switches.

Solution Algorithms for Single-Machine Group Scheduling with Learning Effect and Convex Resource Allocation

Threshold-Based Algorithms for an Online Rolling Horizon Framework Under Uncertainty -- With an Application to Energy Management

Scheduling Real-Time Parallel Applications in Cloud to Minimize Energy Consumption

A Two-stage Framework and Reinforcement Learning-based Optimization Algorithms for Complex Scheduling Problems

Energy-efficient Task Scheduling on Heterogeneous Computing Systems by Linear Programming

Real-time scheduling for parallel tasks with resource reclamation

A Direct Approach for Solving Cloud Computing Task Assignment with Soft Deadlines

Dynamic Task Scheduler for Real Time Requirement in Cloud Computing System.

Hypergraph-partitioning-based online joint scheduling of tasks and data

Optimally rescheduling jobs with a LIFO buffer

Energy-Efficient Iterative Greedy Algorithm for the Distributed Hybrid Flow Shop Scheduling With Blocking Constraints