Abstract:To benefit from the accurate simulation and high-throughput data contributed by advanced digital twin technologies in modern smart plants, the deep reinforcement learning (DRL) method is an appropriate choice to generate a self-optimizing scheduling policy. This study employs the deep Q-network (DQN), which is a successful DRL method, to solve the dynamic scheduling problem of flexible manufacturing systems (FMSs) involving shared resources, route flexibility, and stochastic arrivals of raw products. To model the system in consideration of both manufacturing efficiency and deadlock avoidance, we use a class of Petri nets combining timed-place Petri nets and a system of simple sequential processes with resources (S3PR), which is named as the timed S3PR. The dynamic scheduling problem of the timed S3PR is defined as a Markov decision process (MDP) that can be solved by the DQN. For constructing deep neural networks to approximate the DQN action-value function that maps the timed S3PR states to scheduling rewards, we innovatively employ a graph convolutional network (GCN) as the timed S3PR state approximator by proposing a novel graph convolution layer called a Petri-net convolution (PNC) layer. The PNC layer uses the input and output matrices of the timed S3PR to compute the propagation of features from places to transitions and from transitions to places, thereby reducing the number of parameters to be trained and ensuring robust convergence of the learning process. Experimental results verify that the proposed DQN with a PNC network can provide better solutions for dynamic scheduling problems in terms of manufacturing performance, computational efficiency, and adaptability compared with heuristic methods and a DQN with basic multilayer perceptrons.

Online-learning task scheduling with GNN-RL scheduler in collaborative edge computing

Edge Computing Empowered Digital Twin: an End-to-End Computing Task Scheduling Approach

Reinforcement Learning Based Online Scheduling of Multiple Workflows in Edge Environment

Dependent Task Offloading in Edge Computing Using GNN and Deep Reinforcement Learning

RLCS: Towards a robust and efficient mobile edge computing resource scheduling and task offloading system based on graph neural network

DGCQN: a RL and GCN combined method for DAG scheduling in edge computing

Digital Twin-Assisted Efficient Reinforcement Learning for Edge Task Scheduling

Decentralized Scheduling for Concurrent Tasks in Mobile Edge Computing Via Deep Reinforcement Learning

Smart scheduling of hanging workshop via digital twin and deep reinforcement learning

Petri-net-based deep reinforcement learning for real-time scheduling of automated manufacturing systems

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

Imitation Learning Enabled Task Scheduling for Online Vehicular Edge Computing

Task Placement and Resource Allocation for Edge Machine Learning: A GNN-based Multi-Agent Reinforcement Learning Paradigm

Petri-net-based dynamic scheduling of flexible manufacturing system via deep reinforcement learning with graph convolutional network

Online Optimization of DNN Inference Network Utility in Collaborative Edge Computing

Dynamic scheduling of tasks in cloud manufacturing with multi-agent reinforcement learning

An efficient scheduling scheme for intelligent driving tasks in a novel vehicle-edge architecture considering mobility and load balancing

Flexible job shop scheduling via deep reinforcement learning with meta-path-based heterogeneous graph neural network

Online simulation task scheduling in cloud manufacturing with cross attention and deep reinforcement learning

Edge Generation Scheduling for DAG Tasks Using Deep Reinforcement Learning

Flexible robotic cell scheduling with graph neural network based deep reinforcement learning