Deep Reinforcement Learning Guided Improvement Heuristic for Job Shop Scheduling

Cong Zhang,Zhiguang Cao,Wen Song,Yaoxin Wu,Jie Zhang

2024-02-14

Abstract:Recent studies in using deep reinforcement learning (DRL) to solve Job-shop scheduling problems (JSSP) focus on construction heuristics. However, their performance is still far from optimality, mainly because the underlying graph representation scheme is unsuitable for modelling partial solutions at each construction step. This paper proposes a novel DRL-guided improvement heuristic for solving JSSP, where graph representation is employed to encode complete solutions. We design a Graph Neural-Network-based representation scheme, consisting of two modules to effectively capture the information of dynamic topology and different types of nodes in graphs encountered during the improvement process. To speed up solution evaluation during improvement, we present a novel message-passing mechanism that can evaluate multiple solutions simultaneously. We prove that the computational complexity of our method scales linearly with problem size. Experiments on classic benchmarks show that the improvement policy learned by our method outperforms state-of-the-art DRL-based methods by a large margin.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper attempts to solve the Job - shop Scheduling Problem (JSSP). Specifically, the existing methods based on Deep Reinforcement Learning (DRL) mainly focus on constructing heuristic algorithms when solving JSSP, but the performance of these methods is still far from optimal. The main reason is that the underlying graph representation scheme is not suitable for modeling partial solutions in each construction step. Therefore, this paper proposes a new DRL - guided improved heuristic algorithm, which encodes the complete solution through graph representation, and designs a representation scheme based on Graph Neural Network (GNN) to effectively capture the dynamic topology of the graph and different types of node information encountered in the improvement process. In addition, in order to accelerate the solution evaluation speed in the improvement process, this paper also proposes a new message - passing mechanism that can evaluate multiple solutions simultaneously. The experimental results show that this method significantly outperforms the existing DRL methods in the classical benchmark tests, and the computational complexity has a linear relationship with the problem scale.

Deep Reinforcement Learning Guided Improvement Heuristic for Job Shop Scheduling

Solving flexible job shop scheduling problems via deep reinforcement learning

Combining Reinforcement Learning Algorithms with Graph Neural Networks to Solve Dynamic Job Shop Scheduling Problems

A Reinforcement Learning Environment For Job-Shop Scheduling

Residual Scheduling: A New Reinforcement Learning Approach to Solving Job Shop Scheduling Problem

An end-to-end deep reinforcement learning method based on graph neural network for distributed job-shop scheduling problem

Gated‐Attention Model with Reinforcement Learning for Solving Dynamic Job Shop Scheduling Problem

Curriculum Learning in Job Shop Scheduling using Reinforcement Learning

Dynamic Job-Shop Scheduling via Graph Attention Networks and Deep Reinforcement Learning

Dynamic job-shop scheduling using graph reinforcement learning with auxiliary strategy

End-to-end Multi-Target Flexible Job Shop Scheduling with Deep Reinforcement Learning

Efficient Multi-Objective Optimization on Dynamic Flexible Job Shop Scheduling Using Deep Reinforcement Learning Approach

Reinforcement Learning with Hierarchical Graph Structure for Flexible Job Shop Scheduling

Hybrid Intelligence for Dynamic Job-Shop Scheduling with Deep Reinforcement Learning and Attention Mechanism

Learning to schedule job-shop problems: representation and policy learning using graph neural network and reinforcement learning

Dynamic job-shop scheduling in smart manufacturing using deep reinforcement learning

A Novel Soft Actor–Critic Framework with Disjunctive Graph Embedding and Autoencoder Mechanism for Job Shop Scheduling Problems

A Deep Reinforcement Learning Method Based on a Transformer Model for the Flexible Job Shop Scheduling Problem

Solving Integrated Process Planning and Scheduling Problem via Graph Neural Network Based Deep Reinforcement Learning

Solving the flexible job-shop scheduling problem through an enhanced deep reinforcement learning approach

Reinforcement Learning as an Improvement Heuristic for Real-World Production Scheduling