Abstract:The Flexible Job Shop Scheduling Problem (FJSP), a classic NP-hard optimization challenge, has a direct impact on manufacturing system efficiency. Considering that the FJSP is more complex than the Job Shop Scheduling Problem (JSSP) due to its involvement of both job and machine selection, we have introduced a collaborative agent reinforcement learning (CARL) architecture to tackle this challenge for the first time. To enhance Co-Markov decision process, we introduced disjunctive graphs for the representation of state features. However, the representation of states and actions often leads to suboptimal solutions due to intricate variability. To achieve superior outcomes, we refined our approach to representing states and actions. During the solving process, we employed Graph Attention Network (GAT) to extract global state information from the disjunctive graph and used a Transformer Encoder to quantitatively capture the competitive relationships among machines. We configured two independent encoder–decoder components for job and machine agents, enabling the generation of two distinct action strategies. Finally, we employed the Soft Actor–Critic (SAC) algorithm and an integrated Deep Q Network (DQN) known as D5QN to train the decision network parameters of job and machine agents. Our experiments revealed that after just one training session, collaborative agents acquired exceptional scheduling strategies. These strategies excel not only in solution quality compared to traditional Priority Dispatching Rules (PDR) but also outperform results achieved by some metaheuristic and reinforcement learning algorithms. Additionally, they exhibit greater speed than OR-Tools. Moreover, the empirical findings on both randomized and benchmark instances underscore the remarkable robustness of our acquired policies in practical, large-scale scenarios. Notably, when confronted with the DPpaulli dataset, characterized by a considerable imbalance between the number of operations and machines, our approach achieved optimality in 11 out of 18 FJSP instances.

An improved Q-learning based rescheduling method for flexible job-shops with machine failures

An Improved Genetic Algorithm with Local Search for Dynamic Job Shop Scheduling Problem.

A Q-Learning Rescheduling Approach to the Flexible Job Shop Problem Combining Energy and Productivity Objectives

Multi-objective reinforcement learning framework for dynamic flexible job shop scheduling problem with uncertain events

Deep reinforcement learning for dynamic flexible job shop scheduling problem considering variable processing times

A genetic algorithm-based approach for flexible job shop rescheduling problem with machine failure interference

Dynamic scheduling for flexible job shop based on MachineRank algorithm and reinforcement learning

Dynamic scheduling for flexible job shop using a deep reinforcement learning approach

A Variable Interval Rescheduling Strategy for Dynamic Flexible Job Shop Scheduling Problem by Improved Genetic Algorithm

Multi-Agent Reinforcement Learning for Extended Flexible Job Shop Scheduling

A multi-agent double Deep-Q-network based on state machine and event stream for flexible job shop scheduling problem

Deep reinforcement learning for adaptive flexible job shop scheduling: coping with variability and uncertainty

Adaptive scheduling for assembly job shop with uncertain assembly times based on dual Q-learning

Dynamic flexible scheduling with transportation constraints by multi-agent reinforcement learning

Deep Reinforcement Learning for Dynamic Flexible Job Shop Scheduling with Random Job Arrival

Q -Learning based multi-objective immune algorithm for fuzzy flexible job shop scheduling problem considering dynamic disruptions

A Reinforcement Learning Approach for Flexible Job Shop Scheduling Problem With Crane Transportation and Setup Times

Efficient Multi-Objective Optimization on Dynamic Flexible Job Shop Scheduling Using Deep Reinforcement Learning Approach

An effective hybrid algorithm for joint scheduling of machines and AGVs in flexible job shop

Dynamic Multi-Objective Scheduling for Flexible Job Shop by Deep Reinforcement Learning.

A novel collaborative agent reinforcement learning framework based on an attention mechanism and disjunctive graph embedding for flexible job shop scheduling problem