Abstract:Distributed manufacturing involving heterogeneous factories presents significant challenges to enterprises. Furthermore, the need to prioritize various jobs based on order urgency and customer importance further complicates the scheduling process. Consequently, this study addresses the practical issue by tackling the distributed heterogeneous hybrid flow shop scheduling problem with multiple priorities of jobs (DHHFSP-MPJ). The primary objective is to simultaneously minimize the total weighted tardiness and total energy consumption. To solve DHHFSP-MPJ, a double deep Q-network-based co-evolution (D2QCE) is developed with four features: i) The global and local searches are allocated into two populations to balance computational resources; ii) A hybrid heuristic strategy is proposed to obtain an initialized population with great convergence and diversity; iii) Four knowledge-based neighborhood structures are proposed to accelerate converging. Next, the double deep Q-Network is applied to learn operator selection; and iv) An energy-efficient strategy is presented to save energy. To verify the effectiveness of D2QCE, five state-of-the-art algorithms are compared on 20 instances and a real-world case. The results of numerical experiments indicate that: i) The D2QN can learn fast by only consuming a few computation resources and can select the best operator. ii) Combining D2QN and co-evolution can vastly improve the performance of evolutionary algorithms for solving distributed shop scheduling. iii) The proposed D2QCE has better performance than state-of-the-arts for DHHFSP-MPJ Note to Practitioners—This paper is inspired by a real-world problem encountered in blanking workshop systems within the manufacturing of large engineering equipment. In this practical scenario, jobs come with varying priorities and distinct due dates. Balancing these priority and due date constraints while efficiently scheduling a considerable volume of jobs to enhance enterprise profitability poses a significant challenge. Thus, this scheduling problem is abstracted to the distributed heterogeneous hybrid flow shop scheduling problem with multiple priorities of jobs. The objectives are minimizing weighted due date delay and total energy consumption. Notably, this model has never been studied before. To address this, we’ve formulated a mixed-integer linear programming model and developed a novel co-evolutionary algorithm based on double deep Q-networks (DQN). Our approach introduces several key components. First, we present a co-evolutionary framework to strike a balance between global and local search aspects. Additionally, we’ve devised three problem-specific enhancement strategies to expedite convergence, which include hybrid initialization, local search techniques, and energy-saving measures. To accelerate the learning process of selecting the optimal operator with minimal computational resources, we employ the double DQN. Experimental results demonstrate the superior performance of our approach, outperforming state-of-the-art algorithms when applied to a real-world case. In summary, this work proposes an extended DHHFSP and provides a case of designing the deep learning-assisted evolutionary algorithm. However, online deep reinforcement learning (DRL) consumes additional time, and the generalization of online DRL needs to be improved. In future research, we will consider the dynamic events such as new jobs insert and due date change for the blanking workshop. Moreover, the end-to-end model will be considered to save energy and realize sustainable DRL.

A Q -learning-based hyper-heuristic evolutionary algorithm for the distributed flexible job-shop scheduling problem with crane transportation

Deep Reinforcement Learning Driven Trajectory-Based Meta-Heuristic for Distributed Heterogeneous Flexible Job Shop Scheduling Problem

Double DQN-Based Coevolution for Green Distributed Heterogeneous Hybrid Flowshop Scheduling With Multiple Priorities of Jobs

A Hybrid Differential Evolution And Estimation Of Distribution Algorithm Based On Neighbourhood Search For Job Shop Scheduling Problems

A Hyperheuristic With Q-Learning for the Multiobjective Energy-Efficient Distributed Blocking Flow Shop Scheduling Problem

A Reinforcement Learning Driven Artificial Bee Colony Algorithm for Distributed Heterogeneous No-Wait Flowshop Scheduling Problem with Sequence-Dependent Setup Times

A Cooperative Learning-Aware Dynamic Hierarchical Hyper-Heuristic for Distributed Heterogeneous Mixed No-Wait Flow-Shop Scheduling

A Bi-Learning Evolutionary Algorithm for Transportation-Constrained and Distributed Energy-Efficient Flexible Scheduling

A Learning-Assisted Bi-Population Evolutionary Algorithm for Distributed Flexible Job-Shop Scheduling with Maintenance Decisions

Evolutionary Computation and Reinforcement Learning Integrated Algorithm for Distributed Heterogeneous Flowshop Scheduling

Dynamic Intelligent Scheduling in Low-Carbon Heterogeneous Distributed Flexible Job Shops with Job Insertions and Transfers

Co-Evolution with Deep Reinforcement Learning for Energy-Aware Distributed Heterogeneous Flexible Job Shop Scheduling

A Reinforcement Learning Driven Cooperative Meta-Heuristic Algorithm for Energy-Efficient Distributed No-Wait Flow-Shop Scheduling with Sequence-Dependent Setup Time

A Reinforcement Learning Approach for Flexible Job Shop Scheduling Problem With Crane Transportation and Setup Times

A hybrid estimation of distribution algorithm for distributed flexible job shop scheduling with crane transportations

Modelling and optimization of distributed assembly hybrid flowshop scheduling problem with transportation resource scheduling

Distributed Flexible Job-Shop Scheduling Problem Based on Hybrid Chemical Reaction Optimization Algorithm.

A Hybrid Iterated Greedy Algorithm for a Crane Transportation Flexible Job Shop Problem

An Enhanced Multi-Objective Evolutionary Algorithm with Reinforcement Learning for Energy-Efficient Scheduling in the Flexible Job Shop

Competitive and Cooperative-Based Strength Pareto Evolutionary Algorithm for Green Distributed Heterogeneous Flow Shop Scheduling

A feedback learning-based selection hyper-heuristic for distributed heterogeneous hybrid blocking flow-shop scheduling problem with flexible assembly and setup time