Abstract:In modern volatile and complex manufacturing environment, dynamic events such as new job insertions and machine breakdowns may randomly occur at any time and different objectives in conflict with each other should be optimized simultaneously, leading to an urgent requirement of real-time multi-objective rescheduling methods that can achieve both time efficiency and solution quality. In this regard, this paper proposes an on-line rescheduling framework named as two-hierarchy deep Q network (THDQN) for the dynamic multi-objective flexible job shop scheduling problem (DMOFJSP) with new job insertions. Two practical objectives including total weighted tardiness and average machine utilization rate are optimized. The THDQN model contains two deep Q network (DQN) based agents. The higher-level DQN is a controller determining the temporary optimization goal for the lower DQN. At each rescheduling point, it takes the current state features as input and chooses a feasible goal to guide the behaviour of the lower DQN. Four different goals corresponding to four different forms of reward functions are suggested, each of which optimizes an indicator of tardiness or machine utilization rate. The lower-level DQN acts as an actuator. It takes the current state features together with the higher optimization goal as input and chooses a proper dispatching rule to achieve the given goal. Six composite dispatching rules are developed to select an available operation and assign it on a feasible machine, which serve as the candidate action set for the lower DQN. A novel training framework based on double DQN (DDQN) is designed. The trained THDQN is compared with each proposed composite dispatching rule, existing well-known dispatching rules as well as other reinforcement learning based scheduling methods on a wide range of test instances. Results of numerical experiments have confirmed both the effectiveness and generality of the proposed THDQN.

Dynamic parallel machine scheduling with mean weighted tardiness objective by Q-Learning

Minimizing Mean Weighted Tardiness in Unrelated Parallel Machine Scheduling with Reinforcement Learning

Parallel machine scheduling minimizing the mean weighted flow time

Q-Learning Based Dynamic Single Machine Scheduling

Dynamic parallel machine scheduling using the learning agent

Single Machine Scheduling with a General Exponential Learning Effect

Dynamic single machine scheduling using Q-learning agent

Parallel Machines Scheduling with Deteriorating and Learning Effects

Dynamic Parallel Machine Scheduling with Random Breakdowns Using the Learning Agent

Digital Twin-Enabled Dynamic Scheduling with Preventive Maintenance Using a Double-Layer Q-learning Algorithm

A revised deep reinforcement learning algorithm for parallel machine scheduling problem under multi-scenario due date constraints

Parallel Machine Scheduling with Learning Effects

SOLUTION ALGORITHMS FOR SINGLE-MACHINE SCHEDULING WITH LEARNING EFFECTS AND EXPONENTIAL PAST-SEQUENCE-DEPENDENT DELIVERY TIMES

Adaptive scheduling for assembly job shop with uncertain assembly times based on dual Q-learning

An improved Q-learning based rescheduling method for flexible job-shops with machine failures

Dynamic Flexible Job-Shop Scheduling by Multi-Agent Reinforcement Learning with Reward-Shaping

Parallel machine scheduling with position-based deterioration and learning effects in an uncertain manufacturing system

Dynamic Multi-Objective Scheduling for Flexible Job Shop by Deep Reinforcement Learning.

Energy-efficient Multi-objective Distributed Assembly Permutation Flowshop Scheduling by Q-learning Based Meta-heuristics

Parallel Machines Scheduling with Learning Effect and Controllable Processing Times

A Q -learning-based hyper-heuristic evolutionary algorithm for the distributed flexible job-shop scheduling problem with crane transportation