Abstract:To benefit from the accurate simulation and high-throughput data contributed by advanced digital twin technologies in modern smart plants, the deep reinforcement learning (DRL) method is an appropriate choice to generate a self-optimizing scheduling policy. This study employs the deep Q-network (DQN), which is a successful DRL method, to solve the dynamic scheduling problem of flexible manufacturing systems (FMSs) involving shared resources, route flexibility, and stochastic arrivals of raw products. To model the system in consideration of both manufacturing efficiency and deadlock avoidance, we use a class of Petri nets combining timed-place Petri nets and a system of simple sequential processes with resources (S3PR), which is named as the timed S3PR. The dynamic scheduling problem of the timed S3PR is defined as a Markov decision process (MDP) that can be solved by the DQN. For constructing deep neural networks to approximate the DQN action-value function that maps the timed S3PR states to scheduling rewards, we innovatively employ a graph convolutional network (GCN) as the timed S3PR state approximator by proposing a novel graph convolution layer called a Petri-net convolution (PNC) layer. The PNC layer uses the input and output matrices of the timed S3PR to compute the propagation of features from places to transitions and from transitions to places, thereby reducing the number of parameters to be trained and ensuring robust convergence of the learning process. Experimental results verify that the proposed DQN with a PNC network can provide better solutions for dynamic scheduling problems in terms of manufacturing performance, computational efficiency, and adaptability compared with heuristic methods and a DQN with basic multilayer perceptrons.

A Continuous Process Model for Real-Time Scheduling Based on Hybrid Petri Net

Scheduling Fmss Based On A Hybrid Petri Net Framework

Modeling, safety verification and optimization of operating procedures in process systems using hybrid Petri nets

Petri Net Based Software Testing Scheduling And Selecting

Unit-Specific Event-Based and Slot-Based Hybrid Model Framework with Hierarchical Structure for Short-Term Scheduling

Distributed Intelligent Scheduling Of Fms

A Combined Model for Supervisory Control of Continuous Processes

Scheduling-event Driven Modeling and Simulation of Material Flow Process for Petrochemical Industries

Petri-net-based deep reinforcement learning for real-time scheduling of automated manufacturing systems

A Two-Step ACO-PSO Approach to Optimize Flexible Jobshop Scheduling of Air Compressor Based on Petri Net Model

Petri Net Model Predictive Control Method for Batch Chemical Systems

Dynamic Scheduling of Batch Processes Based on Simulation of Petri Nets

A New Simple Effective Continuous-time Model for Scheduling of General Batch Plants.

Optimal Scheduling of Semiconductor Manufacture System Based on Improved Petri Net Model

A Petri Net and Extended Genetic Algorithm Combined Scheduling Method for Wafer Fabrication

A Petri Net Method for the Optimal Job Planning in Assembly Shops

An ANN-Based Approach for Real-Time Scheduling in Cloud Manufacturing

Hybrid Petri Nets and Exploration of It′s Application

Petri-net-based dynamic scheduling of flexible manufacturing system via deep reinforcement learning with graph convolutional network

Scheduling FMS Problems with Heuristic Search Function and Transition-Timed Petri Nets

Plant-wide optimal scheduling of multi-grade PET production with time window constraints: A hybrid discrete/continuous-time optimization formulation