Abstract:The Flexible Job Shop Scheduling Problem (FJSP), a classic NP-hard optimization challenge, has a direct impact on manufacturing system efficiency. Considering that the FJSP is more complex than the Job Shop Scheduling Problem (JSSP) due to its involvement of both job and machine selection, we have introduced a collaborative agent reinforcement learning (CARL) architecture to tackle this challenge for the first time. To enhance Co-Markov decision process, we introduced disjunctive graphs for the representation of state features. However, the representation of states and actions often leads to suboptimal solutions due to intricate variability. To achieve superior outcomes, we refined our approach to representing states and actions. During the solving process, we employed Graph Attention Network (GAT) to extract global state information from the disjunctive graph and used a Transformer Encoder to quantitatively capture the competitive relationships among machines. We configured two independent encoder–decoder components for job and machine agents, enabling the generation of two distinct action strategies. Finally, we employed the Soft Actor–Critic (SAC) algorithm and an integrated Deep Q Network (DQN) known as D5QN to train the decision network parameters of job and machine agents. Our experiments revealed that after just one training session, collaborative agents acquired exceptional scheduling strategies. These strategies excel not only in solution quality compared to traditional Priority Dispatching Rules (PDR) but also outperform results achieved by some metaheuristic and reinforcement learning algorithms. Additionally, they exhibit greater speed than OR-Tools. Moreover, the empirical findings on both randomized and benchmark instances underscore the remarkable robustness of our acquired policies in practical, large-scale scenarios. Notably, when confronted with the DPpaulli dataset, characterized by a considerable imbalance between the number of operations and machines, our approach achieved optimality in 11 out of 18 FJSP instances.

Integrating multi-agent reinforcement learning and 3D A* search for facility layout problem considering connector-assembly

Learning to Cooperate: Application of Deep Reinforcement Learning for Online AGV Path Finding.

Cooperative Multiagent Reinforcement Learning Coupled With A* Search for Ship Multicabin Equipment Layout Considering Pipe Route

Multi agent reinforcement learning for online layout planning and scheduling in flexible assembly systems

Multi-Agent Reinforcement Learning of 3D Furniture Layout Simulation in Indoor Graphics Scenes

Multi-Agent Reinforcement Learning for Real-Time Dynamic Production Scheduling in a Robot Assembly Cell

Swap-based Deep Reinforcement Learning for Facility Location Problems in Networks

A coordinated scheduling approach for task assignment and multi-agent path planning

A Q-learning artificial bee colony for distributed assembly flow shop scheduling with factory eligibility, transportation capacity and setup time

A novel collaborative agent reinforcement learning framework based on an attention mechanism and disjunctive graph embedding for flexible job shop scheduling problem

Workshop Facility Layout Optimization Based on Deep Reinforcement Learning

An adaptive artificial bee colony with reinforcement learning for distributed three-stage assembly scheduling with maintenance

Deep reinforcement learning for layout planning - An MDP-based approach for the facility layout problem

Adaptive Individual Q-Learning-A Multiagent Reinforcement Learning Method for Coordination Optimization

A reinforcement learning-based approach for solving multi-agent job shop scheduling problem

New hybrid algorithm combining multiple transportation modes for an environmental protection workshop layout

Solving dynamic double row layout problem via combining simulated annealing and mathematical programming

Multi-Robot Coordination and Layout Design for Automated Warehousing

DeepMAG: Deep reinforcement learning with multi-agent graphs for flexible job shop scheduling

Cooperative multi-agent reinforcement learning for multi-area integrated scheduling in wafer fabs

LNS2+RL: Combining Multi-agent Reinforcement Learning with Large Neighborhood Search in Multi-agent Path Finding