Abstract:The parallel machine scheduling problem (PMSP) involves the optimized assignment of a set of jobs to a collection of parallel machines, which is a proper formulation for the modern manufacturing environment. Deep reinforcement learning (DRL) has been widely employed to solve PMSP. However, the majority of existing DRL-based frameworks still suffer from generalizability and scalability. More specifically, the state and action design still heavily rely on human efforts. To bridge these gaps, we propose a practical reinforcement learning-based framework to tackle a PMSP with new job arrivals and family setup constraints. We design a variable-length state matrix containing full job and machine information. This enables the DRL agent to autonomously extract features from raw data and make decisions with a global perspective. To efficiently process this novel state matrix, we elaborately modify a Transformer model to represent the DRL agent. By integrating the modified Transformer model to represent the DRL agent, a novel state representation can be effectively leveraged. This innovative DRL framework offers a high-quality and robust solution that significantly reduces the reliance on manual effort traditionally required in scheduling tasks. In the numerical experiment, the stability of the proposed agent during training is first demonstrated. Then we compare this trained agent on 192 instances with several existing approaches, namely a DRL-based approach, a metaheuristic algorithm, and a dispatching rule. The extensive experimental results demonstrate the scalability of our approach and its effectiveness across a variety of scheduling scenarios. Conclusively, our approach can thus solve the scheduling problems with high efficiency and flexibility, paving the way for application of DRL in solving complex and dynamic scheduling problems.

A Parallel Scheduling Algorithm for Reinforcement Learning in Large State Space

Cooperative Modular Reinforcement Learning for Large Discrete Action Space Problem.

Reinforcement Learning for Adaptive Resource Scheduling in Complex System Environments

GARLSched: Generative Adversarial Deep Reinforcement Learning Task Scheduling Optimization for Large-Scale High Performance Computing Systems

A two-stage RNN-based deep reinforcement learning approach for solving the parallel machine scheduling problem with due dates and family setups

A transformer-based deep reinforcement learning approach for dynamic parallel machine scheduling problem with family setups

A2C-DRL: Dynamic Scheduling for Stochastic Edge-Cloud Environments Using A2C and Deep Reinforcement Learning

Effective Multi-User Delay-Constrained Scheduling with Deep Recurrent Reinforcement Learning

A Scheduling Algorithm Based on Reinforcement Learning for Heterogeneous Environments.

Curriculum Learning in Job Shop Scheduling using Reinforcement Learning

Large-scale Machine Learning Cluster Scheduling via Multi-agent Graph Reinforcement Learning

An Effective Asynchronous Framework for Small Scale Reinforcement Learning Problems

An efficient and adaptive design of reinforcement learning environment to solve job shop scheduling problem with soft actor-critic algorithm

Solving non-permutation flow-shop scheduling problem via a novel deep reinforcement learning approach

Minimizing Mean Weighted Tardiness in Unrelated Parallel Machine Scheduling with Reinforcement Learning

Multi-User Delay-Constrained Scheduling With Deep Recurrent Reinforcement Learning

Developing Real-Time Scheduling Policy by Deep Reinforcement Learning

Accelerating Exact Combinatorial Optimization via RL-based Initialization -- A Case Study in Scheduling

Online Control of Adaptive Large Neighborhood Search using Deep Reinforcement Learning

SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores

Multi-Objective Order Scheduling via Reinforcement Learning