Abstract:The Flexible Job Shop Scheduling Problem (FJSP), a classic NP-hard optimization challenge, has a direct impact on manufacturing system efficiency. Considering that the FJSP is more complex than the Job Shop Scheduling Problem (JSSP) due to its involvement of both job and machine selection, we have introduced a collaborative agent reinforcement learning (CARL) architecture to tackle this challenge for the first time. To enhance Co-Markov decision process, we introduced disjunctive graphs for the representation of state features. However, the representation of states and actions often leads to suboptimal solutions due to intricate variability. To achieve superior outcomes, we refined our approach to representing states and actions. During the solving process, we employed Graph Attention Network (GAT) to extract global state information from the disjunctive graph and used a Transformer Encoder to quantitatively capture the competitive relationships among machines. We configured two independent encoder–decoder components for job and machine agents, enabling the generation of two distinct action strategies. Finally, we employed the Soft Actor–Critic (SAC) algorithm and an integrated Deep Q Network (DQN) known as D5QN to train the decision network parameters of job and machine agents. Our experiments revealed that after just one training session, collaborative agents acquired exceptional scheduling strategies. These strategies excel not only in solution quality compared to traditional Priority Dispatching Rules (PDR) but also outperform results achieved by some metaheuristic and reinforcement learning algorithms. Additionally, they exhibit greater speed than OR-Tools. Moreover, the empirical findings on both randomized and benchmark instances underscore the remarkable robustness of our acquired policies in practical, large-scale scenarios. Notably, when confronted with the DPpaulli dataset, characterized by a considerable imbalance between the number of operations and machines, our approach achieved optimality in 11 out of 18 FJSP instances.

Reinforcement Learning Strategy for Solving the Resource-Constrained Project Scheduling Problem by a Team of A-Teams

Reinforcement Learning Strategy for A-Team Solving the Resource-Constrained Project Scheduling Problem

Deep reinforcement learning for solving resource constrained project scheduling problems with resource disruptions

An Agent-Based Dynamic Scheduling Solution for Resource-Constrained Project Scheduling

A joint work package sizing and scheduling problem considering resource constraints with a look-ahead heterogeneous reinforcement learning method

Multi-Task Multi-Agent Reinforcement Learning for Real-Time Scheduling of a Dual-Resource Flexible Job Shop with Robots

A2C-DRL: Dynamic Scheduling for Stochastic Edge-Cloud Environments Using A2C and Deep Reinforcement Learning

A Reinforcement Learning Based Approach For Multi-Projects Scheduling In Cloud Manufacturing

A Multiagent Evolutionary Algorithm for the Resource-Constrained Project Portfolio Selection and Scheduling Problem

Dynamic scheduling of decentralized high-end equipment R&D projects via deep reinforcement learning

Solving job scheduling problems in a resource preemption environment with multi-agent reinforcement learning

An Agile Adaptive Biased-Randomized Discrete-Event Heuristic for the Resource-Constrained Project Scheduling Problem

A co-evolutionary teaching-learning-based optimization algorithm for stochastic RCPSP

New strategies for stochastic resource-constrained project scheduling

Multi-Agent Reinforcement Learning for Real-Time Dynamic Production Scheduling in a Robot Assembly Cell

Curriculum Learning in Job Shop Scheduling using Reinforcement Learning

A novel collaborative agent reinforcement learning framework based on an attention mechanism and disjunctive graph embedding for flexible job shop scheduling problem

Prac: A Software System To Solve The Resource-Constrained Project Scheduling Problem

Deep Reinforcement Learning for the Agile Earth Observation Satellite Scheduling Problem

Simulation of autonomous resource allocation through deep reinforcement learning-based portfolio-project integration

An efficient and adaptive design of reinforcement learning environment to solve job shop scheduling problem with soft actor-critic algorithm