Abstract:The Flexible Job Shop Scheduling Problem (FJSP), a classic NP-hard optimization challenge, has a direct impact on manufacturing system efficiency. Considering that the FJSP is more complex than the Job Shop Scheduling Problem (JSSP) due to its involvement of both job and machine selection, we have introduced a collaborative agent reinforcement learning (CARL) architecture to tackle this challenge for the first time. To enhance Co-Markov decision process, we introduced disjunctive graphs for the representation of state features. However, the representation of states and actions often leads to suboptimal solutions due to intricate variability. To achieve superior outcomes, we refined our approach to representing states and actions. During the solving process, we employed Graph Attention Network (GAT) to extract global state information from the disjunctive graph and used a Transformer Encoder to quantitatively capture the competitive relationships among machines. We configured two independent encoder–decoder components for job and machine agents, enabling the generation of two distinct action strategies. Finally, we employed the Soft Actor–Critic (SAC) algorithm and an integrated Deep Q Network (DQN) known as D5QN to train the decision network parameters of job and machine agents. Our experiments revealed that after just one training session, collaborative agents acquired exceptional scheduling strategies. These strategies excel not only in solution quality compared to traditional Priority Dispatching Rules (PDR) but also outperform results achieved by some metaheuristic and reinforcement learning algorithms. Additionally, they exhibit greater speed than OR-Tools. Moreover, the empirical findings on both randomized and benchmark instances underscore the remarkable robustness of our acquired policies in practical, large-scale scenarios. Notably, when confronted with the DPpaulli dataset, characterized by a considerable imbalance between the number of operations and machines, our approach achieved optimality in 11 out of 18 FJSP instances.

Solving the flow-shop scheduling problem with human factors and two competing agents with deep reinforcement learning

A reinforcement learning-based approach for solving multi-agent job shop scheduling problem

Integration of deep reinforcement learning and multi-agent system for dynamic scheduling of re-entrant hybrid flow shop considering worker fatigue and skill levels

Intelligent Decision-Making of Scheduling for Dynamic Permutation Flowshop via Deep Reinforcement Learning

Solving flexible job shop scheduling problems via deep reinforcement learning

A Deep Multi-Agent Reinforcement Learning Approach to Solve Dynamic Job Shop Scheduling Problem

Dynamic flexible scheduling with transportation constraints by multi-agent reinforcement learning

A Deep Reinforcement Learning Method Based on a Transformer Model for the Flexible Job Shop Scheduling Problem

Solving non-permutation flow-shop scheduling problem via a novel deep reinforcement learning approach

Efficient Multi-Objective Optimization on Dynamic Flexible Job Shop Scheduling Using Deep Reinforcement Learning Approach

Solving job scheduling problems in a resource preemption environment with multi-agent reinforcement learning

Solving task scheduling problems in cloud manufacturing via attention mechanism and deep reinforcement learning

End-to-end Multi-Target Flexible Job Shop Scheduling with Deep Reinforcement Learning

Hierarchical Reinforcement Learning for Multi-Objective Real-Time Flexible Scheduling in a Smart Shop Floor

A novel collaborative agent reinforcement learning framework based on an attention mechanism and disjunctive graph embedding for flexible job shop scheduling problem

Multi-Agent Reinforcement Learning for Real-Time Dynamic Production Scheduling in a Robot Assembly Cell

Dynamic job shop scheduling based on deep reinforcement learning for multi-agent manufacturing systems

A transformer-based deep reinforcement learning approach for dynamic parallel machine scheduling problem with family setups

Deep Reinforcement Learning for Dynamic Flexible Job Shop Scheduling with Random Job Arrival

Evolutionary Computation and Reinforcement Learning Integrated Algorithm for Distributed Heterogeneous Flowshop Scheduling

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments