Abstract:In modern volatile and complex manufacturing environment, dynamic events such as new job insertions and machine breakdowns may randomly occur at any time and different objectives in conflict with each other should be optimized simultaneously, leading to an urgent requirement of real-time multi-objective rescheduling methods that can achieve both time efficiency and solution quality. In this regard, this paper proposes an on-line rescheduling framework named as two-hierarchy deep Q network (THDQN) for the dynamic multi-objective flexible job shop scheduling problem (DMOFJSP) with new job insertions. Two practical objectives including total weighted tardiness and average machine utilization rate are optimized. The THDQN model contains two deep Q network (DQN) based agents. The higher-level DQN is a controller determining the temporary optimization goal for the lower DQN. At each rescheduling point, it takes the current state features as input and chooses a feasible goal to guide the behaviour of the lower DQN. Four different goals corresponding to four different forms of reward functions are suggested, each of which optimizes an indicator of tardiness or machine utilization rate. The lower-level DQN acts as an actuator. It takes the current state features together with the higher optimization goal as input and chooses a proper dispatching rule to achieve the given goal. Six composite dispatching rules are developed to select an available operation and assign it on a feasible machine, which serve as the candidate action set for the lower DQN. A novel training framework based on double DQN (DDQN) is designed. The trained THDQN is compared with each proposed composite dispatching rule, existing well-known dispatching rules as well as other reinforcement learning based scheduling methods on a wide range of test instances. Results of numerical experiments have confirmed both the effectiveness and generality of the proposed THDQN.

Dynamic scheduling with fuzzy clustering based Q-learning

Clustering state membership-based Q-learning for dynamic scheduling

Multi-Objective Fuzzy Flexible Jobshop of Dynamic Scheduling Optimization Based on Hybrid-Nondominanted Sorting Algorithm

Dynamic parallel machine scheduling using the learning agent

Q-Learning Based Dynamic Single Machine Scheduling

Cloud Scheduling Algorithm Based on Fuzzy Clustering

A Distributed Resource Scheduling Based on Fuzzy Cognitive Map for Dynamic Grid Computation Systems

Dynamic flexible scheduling with transportation constraints by multi-agent reinforcement learning

Dynamic Parallel Machine Scheduling with Random Breakdowns Using the Learning Agent

Research on Multi-AGVs dynamic scheduling based on deep reinforcement learning

An improved Q-learning based rescheduling method for flexible job-shops with machine failures

A multi-agent double Deep-Q-network based on state machine and event stream for flexible job shop scheduling problem

Novel approach to intelligent scheduling based on fuzzy feature selection and fuzzy classifier

Q -Learning based multi-objective immune algorithm for fuzzy flexible job shop scheduling problem considering dynamic disruptions

Dynamic parallel machine scheduling with mean weighted tardiness objective by Q-Learning

Dynamic Multi-Objective Scheduling for Flexible Job Shop by Deep Reinforcement Learning.

An Auction-Based Approach for Multi-Agent Uniform Parallel Machine Scheduling with Dynamic Jobs Arrival

Integrated scheduling optimization of U-shaped automated container terminal under loading and unloading mode

Multi-Agent Reinforcement Learning for Extended Flexible Job Shop Scheduling

A Deep Multi-Agent Reinforcement Learning Approach to Solve Dynamic Job Shop Scheduling Problem

Multi-objective reinforcement learning framework for dynamic flexible job shop scheduling problem with uncertain events