Petri-net-based deep reinforcement learning for real-time scheduling of automated manufacturing systems
Jiliang Luo,Sijia Yi,Zexuan Lin,Hongbin Zhang,Jiazhong Zhou
DOI: https://doi.org/10.1016/j.jmsy.2024.05.006
IF: 12.1
2024-05-26
Journal of Manufacturing Systems
Abstract:This work focuses on real-time and joint optimizing issues to handle the routing of automated guided vehicles and allocating processing tasks for automated manufacturing systems. A real-time scheduling approach is proposed based on Petri nets and deep reinforcement learning. First, the considered system is modeled with a placed-timed Petri net, which plays the role of the environment for deep reinforcement learning. Second, a novel graph convolutional network (GCN), called Petri-net-GCN, is designed by using the topological structures of a Petri net model such that it can adapt to changes in Petri-net structures caused by abrupt events such as new orders and failures of devices. Third, a deep Q-network method is presented to train Petri-net-GCN to predict the minimal time to lead a Petri net to its goal if an action (transition) is taken at a present state. Consequently, Petri-net-GCN can replace scheduling rules to schedule an automated manufacturing system in real time. Finally, numerical experiments are carried out where there exist failures of devices and new orders. The results show that Petri-net-GCN can adapt to the Petri-net changes caused by abrupt events and performs much better than the first-in-first-out and longest-processing-time-first rules, which are applicable for real-time scheduling issues in practice.
engineering, manufacturing, industrial,operations research & management science