Abstract:Due to the influence of dynamic changes in the manufacturing environment, a single dispatching rule (SDR) cannot consistently attain better results than other rules for dynamic scheduling problems. Dynamic selection of the most appropriate rule from several SDRs based on the Deep Q-Network (DQN) offers better scheduling performance than using an individual SDR. However, the discreteness of action space caused by the DQN and the simplicity of the action as an SDR limit the selection range and restrict performance improvement. Thus, in this paper, we propose a scheduling method based on deep reinforcement learning for the dynamic flexible job-shop scheduling problem (DFJSP), aiming to minimize the mean tardiness. Firstly, a Markov decision process with composite scheduling action is provided to elaborate the flexible job-shop dynamic scheduling process and transform the DFJSP into an RL task. Subsequently, a composite scheduling action aggregated by SDRs and continuous weight variables is designed to provide a continuous rule space and SDR weight selection. Moreover, a reward function related to mean tardiness performance criteria is designed such that maximizing the cumulative reward is equivalent to minimizing the mean tardiness. Finally, a policy network with states as inputs and weights as outputs is constructed to generate the scheduling decision at each decision point. Also, the deep deterministic policy gradient (DDPG) algorithm is used to train the policy network to select the most appropriate weights at each decision point, thereby aggregating the SDRs into a better rule. Results from numerical experiments reveal that the proposed scheduling method achieves significantly better scheduling results than an SDR and the DQN-based method in dynamically changeable manufacturing environments.

Adaptive scheduling for assembly job shop with uncertain assembly times based on dual Q-learning

Adaptive scheduling of aero-engine assembly based on q-learning in knowledgeable manufacture

A Dual-System Reinforcement Learning Method for Flexible Job Shop Dynamic Scheduling

A multi-level action coupling reinforcement learning approach for online two-stage flexible assembly flow shop scheduling

Dynamic scheduling for flexible job shop using a deep reinforcement learning approach

Multi-objective reinforcement learning framework for dynamic flexible job shop scheduling problem with uncertain events

Deep reinforcement learning for adaptive flexible job shop scheduling: coping with variability and uncertainty

An improved Q-learning based rescheduling method for flexible job-shops with machine failures

Learning to schedule (L2S): adaptive job shop scheduling using double deep Q network

Multi-Agent Reinforcement Learning for Real-Time Dynamic Production Scheduling in a Robot Assembly Cell

Deep Reinforcement Learning for Dynamic Flexible Job Shop Scheduling with Random Job Arrival

Multi-Agent Reinforcement Learning for Extended Flexible Job Shop Scheduling

A Q-learning artificial bee colony for distributed assembly flow shop scheduling with factory eligibility, transportation capacity and setup time

Hierarchical Reinforcement Learning for Multi-Objective Real-Time Flexible Scheduling in a Smart Shop Floor

A Reinforcement Learning Approach for Flexible Job Shop Scheduling Problem With Crane Transportation and Setup Times

A novel method-based reinforcement learning with deep temporal difference network for flexible double shop scheduling problem

A novel shuffled frog-leaping algorithm with reinforcement learning for distributed assembly hybrid flow shop scheduling

Dynamic scheduling for flexible job shop based on MachineRank algorithm and reinforcement learning

Dynamic Flexible Job-Shop Scheduling by Multi-Agent Reinforcement Learning with Reward-Shaping

Digital Twin-Enabled Dynamic Scheduling with Preventive Maintenance Using a Double-Layer Q-learning Algorithm

Deep reinforcement learning for dynamic scheduling of a flexible job shop