Abstract:The Flexible Job Shop Scheduling Problem (FJSP), a classic NP-hard optimization challenge, has a direct impact on manufacturing system efficiency. Considering that the FJSP is more complex than the Job Shop Scheduling Problem (JSSP) due to its involvement of both job and machine selection, we have introduced a collaborative agent reinforcement learning (CARL) architecture to tackle this challenge for the first time. To enhance Co-Markov decision process, we introduced disjunctive graphs for the representation of state features. However, the representation of states and actions often leads to suboptimal solutions due to intricate variability. To achieve superior outcomes, we refined our approach to representing states and actions. During the solving process, we employed Graph Attention Network (GAT) to extract global state information from the disjunctive graph and used a Transformer Encoder to quantitatively capture the competitive relationships among machines. We configured two independent encoder–decoder components for job and machine agents, enabling the generation of two distinct action strategies. Finally, we employed the Soft Actor–Critic (SAC) algorithm and an integrated Deep Q Network (DQN) known as D5QN to train the decision network parameters of job and machine agents. Our experiments revealed that after just one training session, collaborative agents acquired exceptional scheduling strategies. These strategies excel not only in solution quality compared to traditional Priority Dispatching Rules (PDR) but also outperform results achieved by some metaheuristic and reinforcement learning algorithms. Additionally, they exhibit greater speed than OR-Tools. Moreover, the empirical findings on both randomized and benchmark instances underscore the remarkable robustness of our acquired policies in practical, large-scale scenarios. Notably, when confronted with the DPpaulli dataset, characterized by a considerable imbalance between the number of operations and machines, our approach achieved optimality in 11 out of 18 FJSP instances.

A Novel Soft Actor–Critic Framework with Disjunctive Graph Embedding and Autoencoder Mechanism for Job Shop Scheduling Problems

A novel collaborative agent reinforcement learning framework based on an attention mechanism and disjunctive graph embedding for flexible job shop scheduling problem

A Novel Reinforcement Learning Approach for Enhancing Flexible Job-Shop Scheduling with Dual Gated-Attention Network

Learning to Dispatch for Job Shop Scheduling Via Deep Reinforcement Learning

End-to-end Multi-Target Flexible Job Shop Scheduling with Deep Reinforcement Learning

Learning to Schedule Job Shop Scheduling Problem with Maintenance Time Using Graph Node Embedding and Deep Reinforcement Learning

Learning to schedule job-shop problems: representation and policy learning using graph neural network and reinforcement learning

Gated‐Attention Model with Reinforcement Learning for Solving Dynamic Job Shop Scheduling Problem

A novel priority dispatch rule generation method based on graph neural network and reinforcement learning for distributed job-shop scheduling

An end-to-end deep reinforcement learning method based on graph neural network for distributed job-shop scheduling problem

Dynamic Job-Shop Scheduling via Graph Attention Networks and Deep Reinforcement Learning

Flexible job shop scheduling via deep reinforcement learning with meta-path-based heterogeneous graph neural network

A Deep Reinforcement Learning Approach to Flexible Job Shop Scheduling

A graph neural networks-based deep Q-learning approach for job shop scheduling problems in traffic management

Solving flexible job shop scheduling problems via deep reinforcement learning

An Effective Multi-Agent-based Graph Reinforcement Learning Method for Solving Flexible Job Shop Scheduling Problem

A multi-action deep reinforcement learning framework for flexible Job-shop scheduling problem

Solving Size-Agnostic Job Shop Scheduling Problems Like GPT Speaks*

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

An actor-critic algorithm with policy gradients to solve the job shop scheduling problem using deep double recurrent agents

Evolution strategies-based optimized graph reinforcement learning for solving dynamic job shop scheduling problem