Abstract:In smart manufacturing systems (SMSs), flexible job-shop scheduling with transportation constraints (FJSPT) is essential to optimize solutions for maximizing productivity, considering production flexibility based on automated guided vehicles (AGVs). Recent developments in deep reinforcement learning (DRL)-based methods for FJSPT have encountered a scale generalization challenge. These methods underperform when applied to environment at scales different from their training set, resulting in low-quality solutions. To address this, we introduce a novel graph-based DRL method, named the Heterogeneous Graph Scheduler (HGS). Our method leverages locally extracted relational knowledge among operations, machines, and vehicle nodes for scheduling, with a graph-structured decision-making framework that reduces encoding complexity and enhances scale generalization. Our performance evaluation, conducted with benchmark datasets, reveals that the proposed method outperforms traditional dispatching rules, meta-heuristics, and existing DRL-based approaches in terms of makespan performance, even on large-scale instances that have not been experienced during training.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the scale generalization problem of flexible job - shop scheduling and transportation constraints (FJSPT) in different - scale environments in intelligent manufacturing systems. Specifically, the existing methods based on deep reinforcement learning (DRL) perform poorly when applied to environments with different scales from the training set, resulting in low - quality solutions. In addition, these methods have technical challenges in handling end - to - end decision - making because they usually rely on rule bases and lack sufficient exploration of the action space. To address these challenges, the paper proposes a new graph - based DRL method - Heterogeneous Graph Scheduler (HGS). HGS schedules by locally extracting the relational knowledge among operation, machine and vehicle nodes, and adopts a graph - structured decision - making framework to reduce encoding complexity and enhance scale generalization ability. The performance evaluation on benchmark datasets in the paper shows that the proposed method outperforms traditional scheduling rules, meta - heuristic methods and existing DRL methods in terms of makespan performance, especially on large - scale instances not experienced during the training process. The main contributions of the paper include: 1. Developed a new heterogeneous graph structure and structure - aware heterogeneous graph encoder specifically designed for FJSPT, enabling each node to aggregate messages from adjacent nodes of different categories, such as operations, machines and vehicles. 2. Constructed a three - stage decoder specifically customized for FJSPT. Through three - stage decoding, the decoder selects operation, machine and vehicle nodes in turn according to the graph embedding obtained from the encoder at each decision - making moment to minimize the makespan. 3. Integrated the encoder, decoder and reinforcement learning framework to develop the HGS module. This method not only surpasses traditional scheduling rules, meta - heuristic methods and existing DRL algorithms in scale generalization ability, but also verifies its superiority through simulations on multiple benchmark datasets. To the best of the authors' knowledge, this is the first scale - independent DRL - based FJSPT scheduler.

Learning-enabled Flexible Job-shop Scheduling for Scalable Smart Manufacturing

Flexible job shop scheduling via deep reinforcement learning with meta-path-based heterogeneous graph neural network

End-to-end Multi-Target Flexible Job Shop Scheduling with Deep Reinforcement Learning

Hierarchical Reinforcement Learning for Multi-Objective Real-Time Flexible Scheduling in a Smart Shop Floor

Solving flexible job shop scheduling problems via deep reinforcement learning

Dynamic Flexible Job-Shop Scheduling by Multi-Agent Reinforcement Learning with Reward-Shaping

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

Deep Reinforcement Learning for Dynamic Flexible Job Shop Scheduling with Random Job Arrival

A Deep Reinforcement Learning Method Based on a Transformer Model for the Flexible Job Shop Scheduling Problem

Solving the flexible job-shop scheduling problem through an enhanced deep reinforcement learning approach

Dynamic scheduling for flexible job shop using a deep reinforcement learning approach

Job shop smart manufacturing scheduling by deep reinforcement learning

A Novel Reinforcement Learning Approach for Enhancing Flexible Job-Shop Scheduling with Dual Gated-Attention Network

Reinforcement Learning with Hierarchical Graph Structure for Flexible Job Shop Scheduling

Dynamic Intelligent Scheduling in Low-Carbon Heterogeneous Distributed Flexible Job Shops with Job Insertions and Transfers

Deep-Reinforcement-Learning-Based Production Scheduling in Industrial Internet of Things

Hybrid Intelligence for Dynamic Job-Shop Scheduling with Deep Reinforcement Learning and Attention Mechanism

A novel collaborative agent reinforcement learning framework based on an attention mechanism and disjunctive graph embedding for flexible job shop scheduling problem

Efficient Multi-Objective Optimization on Dynamic Flexible Job Shop Scheduling Using Deep Reinforcement Learning Approach

Dynamic job-shop scheduling in smart manufacturing using deep reinforcement learning

Residual Scheduling: A New Reinforcement Learning Approach to Solving Job Shop Scheduling Problem