Abstract:The dynamic job-shop scheduling problem is a complex and uncertain task that involves optimizing production planning and resource allocation in a dynamic production environment. Traditional methods are limited in effectively handling dynamic events and quickly generating scheduling solutions; in order to solve this problem, this paper proposes a solution by transforming the dynamic job-shop scheduling problem into a Markov decision process and leveraging deep reinforcement learning techniques. The proposed framework introduces several innovative components, which make full use of human domain knowledge and machine computing power, to realize the goal of man–machine collaborative decision-making. Firstly, we utilize disjunctive graphs as the state representation, capturing the complex relationships between various elements of the scheduling problem. Secondly, we select a set of dispatching rules through data envelopment analysis to form the action space, allowing for flexible and efficient scheduling decisions. Thirdly, the transformer model is employed as the feature extraction module, enabling effective capturing of state relationships and improving the representation power. Moreover, the framework incorporates the dueling double deep Q-network with prioritized experience replay, mapping each state to the most appropriate dispatching rule. Additionally, a dynamic target strategy with an elite mechanism is proposed. Through extensive experiments conducted on multiple examples, our proposed framework consistently outperformed traditional dispatching rules, genetic algorithms, and other reinforcement learning methods, achieving improvements of 15.98%, 17.98%, and 13.84%, respectively. These results validate the effectiveness and superiority of our approach in addressing dynamic job-shop scheduling problems.

An End-to-end Scheduling Digital Twin for Multistage Batch Plants Considering Safety

A novel hybrid differential evolution approach to scheduling of large-scale zero-wait batch processes with setup times

Digital twin and deep reinforcement learning enabled real-time scheduling for complex product flexible shop-floor

Edge Computing Empowered Digital Twin: an End-to-End Computing Task Scheduling Approach

Smart scheduling of hanging workshop via digital twin and deep reinforcement learning

Deep-Reinforcement-Learning-Based Production Scheduling in Industrial Internet of Things

A New Simple Effective Continuous-time Model for Scheduling of General Batch Plants.

A Decomposition-Based Approach for Multitask Scheduling with Execution Uncertainty in Industrial Internet of Things

Field-synchronized Digital Twin framework for production scheduling with uncertainty

Digital Twin Enhanced Dynamic Job-Shop Scheduling

Digital twin oriented multi-objective flexible job shop scheduling model and its hybrid particle swarm optimization

A Digital Twin-Based Automatic Programming Method for Adaptive Control of Manufacturing Cells

Digital-twin-based job shop multi-objective scheduling model and strategy

Stochastic programming-based mathematical model and solution strategy for chemical production scheduling with processing time uncertainty

A digital twin-based decision support approach for AGV scheduling

A digital twin-driven flexible scheduling method in a human–machine collaborative workshop based on hierarchical reinforcement learning

A Deep Multi-Agent Reinforcement Learning Approach to Solve Dynamic Job Shop Scheduling Problem

Dynamic Job-Shop Scheduling Based on Transformer and Deep Reinforcement Learning

A simulation-based digital twin for data-driven maintenance scheduling of risk-prone production lines via actor critics

Job Shop Scheduling Based on Digital Twin Technology: A Survey and an Intelligent Platform

Deep reinforcement learning for dynamic scheduling of a flexible job shop