Abstract:To benefit from the accurate simulation and high-throughput data contributed by advanced digital twin technologies in modern smart plants, the deep reinforcement learning (DRL) method is an appropriate choice to generate a self-optimizing scheduling policy. This study employs the deep Q-network (DQN), which is a successful DRL method, to solve the dynamic scheduling problem of flexible manufacturing systems (FMSs) involving shared resources, route flexibility, and stochastic arrivals of raw products. To model the system in consideration of both manufacturing efficiency and deadlock avoidance, we use a class of Petri nets combining timed-place Petri nets and a system of simple sequential processes with resources (S3PR), which is named as the timed S3PR. The dynamic scheduling problem of the timed S3PR is defined as a Markov decision process (MDP) that can be solved by the DQN. For constructing deep neural networks to approximate the DQN action-value function that maps the timed S3PR states to scheduling rewards, we innovatively employ a graph convolutional network (GCN) as the timed S3PR state approximator by proposing a novel graph convolution layer called a Petri-net convolution (PNC) layer. The PNC layer uses the input and output matrices of the timed S3PR to compute the propagation of features from places to transitions and from transitions to places, thereby reducing the number of parameters to be trained and ensuring robust convergence of the learning process. Experimental results verify that the proposed DQN with a PNC network can provide better solutions for dynamic scheduling problems in terms of manufacturing performance, computational efficiency, and adaptability compared with heuristic methods and a DQN with basic multilayer perceptrons.

Distributed Intelligent Scheduling Of Fms

A New Flexible Multi-Agvs Scheduling Model with Pre-Waiting Time in an Intelligent Manufacturing System.

Scheduling Fmss Based On A Hybrid Petri Net Framework

Real-time scheduling scheme for EPA networks

A New Method For Solving Scheduling Problems In Manufacturing Environments

FBS-enhanced Agent-Based Dynamic Scheduling in FMS.

An Estimation of Distribution Algorithm for Scheduling Problem of Flexible Manufacturing Systems Using Petri Nets

Scheduling FMS Problems with Heuristic Search Function and Transition-Timed Petri Nets

A New Flexible Approach for Flexible Scheduling System

Scheduling optimization for FMS based on Petri net modeling and GA

Modeling and control of workstation level information flow in FMS using modified Petri nets

Dynamic scheduling in RFID-driven discrete manufacturing system by using multi-layer network metrics as heuristic information

Deadlock-free Scheduling for Flexible Manufacturing Systems Using Petri Nets and Heuristic Search.

Cross-Network Fusion and Scheduling for Heterogeneous Networks in Smart Factory

Modelling and Simulation of an Intelligent Flexible Manufacturing System Via High-Level Object Petri Nets (HLOPN)

Simulation and Real-Time Control of Fms with A High-Level Petri Net Model

Hybrid Heuristic Search Approach for Deadlock-Free Scheduling of Flexible Manufacturing Systems Using Petri Nets

Scheduling of Flexible Manufacturing Systems Subject to No-Wait Constraints Via Petri Nets and Heuristic Search

Petri-net-based dynamic scheduling of flexible manufacturing system via deep reinforcement learning with graph convolutional network

Deadlock-free Scheduling Method for Automated Manufacturing Systems Using Genetic Algorithm and Petri Nets.

A Hybrid Estimation-of-Distribution Algorithm for Scheduling Flexible Job Shop with Limited Buffers Based on Petri Nets