Abstract:Distributed manufacturing can reduce the production cost through the cooperation among factories, and it has been an important trend in the industrial field. For the enterprises with daily delivered production tasks, the random job arrivals are regular. Thus, the Distributed Job-shop Scheduling Problem (DJSP) with random job arrivals is studied, and it is a typical case from the equipment manufacturing industry. The DJSP involves two coupled decision-making processes, job assigning and job sequencing, and the distributed and uncertain production environment requires the scheduling method to be more responsive and adaptive. Thus, a Deep Reinforcement Learning (DRL) based multi-agent method is explored, and it is composed of the assigning agent and the sequencing agent. Two Markov Decision Processes (MDPs) are formulated for the two agents respectively. In the MDP for the assigning agent, fourteen factory-and-job related features are extracted as the state features, seven composite assigning rules are designed as the candidate actions, and the reward depends on the total processing time of different factories. In the MDP of the sequencing agent, five machine-and-job related features are set as the state features, six sequencing rules make up the action space, and the change of the factory makespan is the reward. Besides, to enhance the learning ability of the agents, a Deep Q-Network (DQN) framework with variable threshold probability in the training stage is designed, which can balance the exploitation and exploration in the model training. The proposed multi-agent method's effectiveness is proved by the independent utility test and the comparison test that are based on 1350 production instances, and its practical value in the actual production is implied by the case study from an automotive engine manufacturing company.

An end-to-end deep reinforcement learning method based on graph neural network for distributed job-shop scheduling problem

A novel priority dispatch rule generation method based on graph neural network and reinforcement learning for distributed job-shop scheduling

A Cooperative Hierarchical Deep Reinforcement Learning based Multi-agent Method for Distributed Job Shop Scheduling Problem with Random Job Arrivals

Combining Reinforcement Learning Algorithms with Graph Neural Networks to Solve Dynamic Job Shop Scheduling Problems

Flexible Job-shop Scheduling Method Based on Deep Reinforcement Learning

End-to-end Multi-Target Flexible Job Shop Scheduling with Deep Reinforcement Learning

Flexible job shop scheduling via deep reinforcement learning with meta-path-based heterogeneous graph neural network

Dynamic Job Shop Scheduling Via Deep Reinforcement Learning

Evolution strategies-based optimized graph reinforcement learning for solving dynamic job shop scheduling problem

A Deep Reinforcement Learning Approach to Flexible Job Shop Scheduling

Solving Integrated Process Planning and Scheduling Problem via Graph Neural Network Based Deep Reinforcement Learning

Flexible Job-Shop Scheduling Via Graph Neural Network and Deep Reinforcement Learning

Dynamic Job-Shop Scheduling via Graph Attention Networks and Deep Reinforcement Learning

A Novel Reinforcement Learning Approach for Enhancing Flexible Job-Shop Scheduling with Dual Gated-Attention Network

Hybrid Intelligence for Dynamic Job-Shop Scheduling with Deep Reinforcement Learning and Attention Mechanism

Learning to schedule job-shop problems: representation and policy learning using graph neural network and reinforcement learning

Dynamic job-shop scheduling using graph reinforcement learning with auxiliary strategy

Learning to Dispatch for Job Shop Scheduling Via Deep Reinforcement Learning

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

A multi-action deep reinforcement learning framework for flexible Job-shop scheduling problem

Deep reinforcement learning for dynamic distributed job shop scheduling problem with transfers