Abstract:In modern volatile and complex manufacturing environment, dynamic events such as new job insertions and machine breakdowns may randomly occur at any time and different objectives in conflict with each other should be optimized simultaneously, leading to an urgent requirement of real-time multi-objective rescheduling methods that can achieve both time efficiency and solution quality. In this regard, this paper proposes an on-line rescheduling framework named as two-hierarchy deep Q network (THDQN) for the dynamic multi-objective flexible job shop scheduling problem (DMOFJSP) with new job insertions. Two practical objectives including total weighted tardiness and average machine utilization rate are optimized. The THDQN model contains two deep Q network (DQN) based agents. The higher-level DQN is a controller determining the temporary optimization goal for the lower DQN. At each rescheduling point, it takes the current state features as input and chooses a feasible goal to guide the behaviour of the lower DQN. Four different goals corresponding to four different forms of reward functions are suggested, each of which optimizes an indicator of tardiness or machine utilization rate. The lower-level DQN acts as an actuator. It takes the current state features together with the higher optimization goal as input and chooses a proper dispatching rule to achieve the given goal. Six composite dispatching rules are developed to select an available operation and assign it on a feasible machine, which serve as the candidate action set for the lower DQN. A novel training framework based on double DQN (DDQN) is designed. The trained THDQN is compared with each proposed composite dispatching rule, existing well-known dispatching rules as well as other reinforcement learning based scheduling methods on a wide range of test instances. Results of numerical experiments have confirmed both the effectiveness and generality of the proposed THDQN.

Clustering state membership-based Q-learning for dynamic scheduling

Dynamic scheduling with fuzzy clustering based Q-learning

Q-Learning Based Dynamic Single Machine Scheduling

Dynamic single machine scheduling using Q-learning agent

A Self-adaptive Knowledge Based Dynamic Scheduling Decision Mechanism

Dynamic parallel machine scheduling using the learning agent

Dynamic parallel machine scheduling with mean weighted tardiness objective by Q-Learning

Knowledge Update in a Knowledge-Based Dynamic Scheduling Decision System

An improved Q-learning based rescheduling method for flexible job-shops with machine failures

An Interoperable Adaptive Scheduling Strategy for Knowledgeable Manufacturing Based on SMGWQ-learning

Dynamic flexible scheduling with transportation constraints by multi-agent reinforcement learning

Dynamic scheduling for flexible job shop using a deep reinforcement learning approach

DQN-based online scheduling algorithm for hybrid flow shop to minimize the total tardiness

Dynamic Multi-Objective Scheduling for Flexible Job Shop by Deep Reinforcement Learning.

Research on Multi-AGVs dynamic scheduling based on deep reinforcement learning

Dynamic Parallel Machine Scheduling with Random Breakdowns Using the Learning Agent

Dynamic Flexible Job-Shop Scheduling by Multi-Agent Reinforcement Learning with Reward-Shaping

A Distributed Q-Learning Algorithm for Multi-Agent Team Coordination

Dynamic scheduling for flexible job shop based on MachineRank algorithm and reinforcement learning

Deep reinforcement learning for dynamic distributed job shop scheduling problem with transfers

A Scheduling Model and Q-learning Algorithm for Yard Trailers at Container Terminals