Abstract:Reinforcement learning-based methods have addressed production scheduling problems with flexible processing constraints. However, delayed rewards arise due to the dynamic arrival of jobs and transportation constraints between two successive operations. The flow time of operations can only be determined after processing due to the possibility that the solution for job sequencing may change if new operations are inserted in dynamic environments. Job sequencing is often overlooked in single-agent-based scheduling methods. The lack of information sharing between multiple agents necessitates that researchers manually design reward functions to fit the relationship between optimization objectives and rewards, thereby reducing the accuracy of the learned policies. Thus, this paper proposes a multi-agent-based scheduling optimization framework that facilitates collaboration between the agents of both machines and jobs to address dynamic flexible job-shop scheduling problems (DFJSP) with transportation time constraints. Then, this paper formulates the Partial Observation Markov Decision Process and constructs a reward-sharing mechanism to tackle the delayed reward issue and facilitate policy learning. Finally, we develop an improved multi-agent dueling double deep Q network algorithm to optimize scheduling policy during long-term training. The results show that, compared with the state-of-the-art methods, the proposed method efficiently shortens the weighted flow time under the trained and unseen scenarios. Additionally, the case study results demonstrate its efficiency and responsiveness. It indicates that the proposed method efficiently addresses production scheduling problems with complex constraints, including the insertion of jobs, transportation time constraints, and flexible processing routes.

Q-Learning Based Dynamic Single Machine Scheduling

Dynamic single machine scheduling using Q-learning agent

Dynamic parallel machine scheduling using the learning agent

Dynamic parallel machine scheduling with mean weighted tardiness objective by Q-Learning

Dynamic scheduling with fuzzy clustering based Q-learning

A Self-adaptive Knowledge Based Dynamic Scheduling Decision Mechanism

Clustering state membership-based Q-learning for dynamic scheduling

Dynamic Parallel Machine Scheduling with Random Breakdowns Using the Learning Agent

An improved Q-learning based rescheduling method for flexible job-shops with machine failures

Digital Twin-Enabled Dynamic Scheduling with Preventive Maintenance Using a Double-Layer Q-learning Algorithm

A Deep Multi-Agent Reinforcement Learning Approach to Solve Dynamic Job Shop Scheduling Problem

Pattern driven scheduling system based on reinforcement learning

Double Deep Q-network-based Self-Adaptive Scheduling Approach for Smart Shop Floor.

Dynamic production scheduling towards self-organizing mass personalization: A multi-agent dueling deep reinforcement learning approach

Knowledge Update in a Knowledge-Based Dynamic Scheduling Decision System

Research on Multi-AGVs dynamic scheduling based on deep reinforcement learning

Dynamic Scheduling Method of Multi-objective Job Shop Based on Reinforcement Learning

Deep reinforcement learning for dynamic scheduling of a flexible job shop

Dynamic flexible scheduling with transportation constraints by multi-agent reinforcement learning

An Interoperable Adaptive Scheduling Strategy for Knowledgeable Manufacturing Based on SMGWQ-learning

Q-Learning Based Optimisation Framework for Real-Time Mixed-Task Scheduling