Abstract:The resource-constrained project scheduling problem (RCPSP) is encountered in many fields, including manufacturing, supply chain, and construction. Nowadays, with the rapidly changing external environment and the emergence of new models such as smart manufacturing, it is more and more necessary to study RCPSP considering resource disruptions. A framework based on reinforcement learning (RL) and graph neural network (GNN) is proposed to solve RCPSP and further solve the RCPSP with resource disruptions (RCPSP-RD) on this basis. The scheduling process is formulated as sequential decision-making problems. Based on that, Markov decision process (MDP) models are developed for RL to learn scheduling policies. A GNN-based structure is proposed to extract features from problems and map them to action probability distributions by policy network. To optimize the scheduling policy, proximal policy optimization (PPO) is applied to train the model end-to-end. Computational results on benchmark instances show that the RL-GNN algorithm achieves competitive performance compared with some widely used methods.

What problem does this paper attempt to address?

### Problems Addressed by the Paper This paper primarily aims to address the Resource-Constrained Project Scheduling Problem (RCPSP) and its extended version under resource disruptions (RCPSP-RD). Specifically: 1. **Resource-Constrained Project Scheduling Problem (RCPSP)**: - **Problem Description**: The paper first focuses on the traditional RCPSP, which aims to minimize the project duration while satisfying precedence constraints and resource availability. - **Methodology**: The problem is tackled by combining Reinforcement Learning (RL) with Graph Neural Networks (GNN). The scheduling process is modeled as a sequential decision problem, constructing a Markov Decision Process (MDP). GNN is used to extract problem features, which are then mapped to action probability distributions through a policy network. 2. **Resource-Constrained Project Scheduling Problem with Resource Disruptions (RCPSP-RD)**: - **Problem Description**: The problem is further extended to consider resource disruptions, i.e., scheduling projects when resource capacities vary over time. - **Methodology**: The paper proposes a framework based on RL and GNN to handle uncertainties brought by resource disruptions. The MDP model is adjusted to accommodate resource changes and activity interruptions, and the Proximal Policy Optimization (PPO) algorithm is used for end-to-end training. ### Main Contributions 1. **Framework Design**: A framework combining RL and GNN is proposed to solve RCPSP and further address the RCPSP-RD problem. The MDP model and GNN structure are used to extract useful features from the problem and map them to action probability distributions. 2. **Experimental Results**: Experiments conducted on the Project Scheduling Problem Library (PSPLIB) show that the algorithm performs well not only on traditional RCPSP but also on RCPSP-RD, demonstrating competitive performance and good generalization capabilities. ### Summary This paper aims to solve the traditional Resource-Constrained Project Scheduling Problem and its extended version under resource disruptions by combining Reinforcement Learning and Graph Neural Networks, thereby improving the robustness and adaptability of scheduling strategies.

Deep reinforcement learning for solving resource constrained project scheduling problems with resource disruptions

Solving Integrated Process Planning and Scheduling Problem via Graph Neural Network Based Deep Reinforcement Learning

An end-to-end deep reinforcement learning method based on graph neural network for distributed job-shop scheduling problem

Flexible job shop scheduling via deep reinforcement learning with meta-path-based heterogeneous graph neural network

Dynamic scheduling of decentralized high-end equipment R&D projects via deep reinforcement learning

Multi-Agent Reinforcement Learning for Job Shop Scheduling in Dynamic Environments

A Reinforcement Learning Based Approach For Multi-Projects Scheduling In Cloud Manufacturing

End-to-end Multi-Target Flexible Job Shop Scheduling with Deep Reinforcement Learning

Learning to schedule job-shop problems: representation and policy learning using graph neural network and reinforcement learning

A two-stage RNN-based deep reinforcement learning approach for solving the parallel machine scheduling problem with due dates and family setups

A novel priority dispatch rule generation method based on graph neural network and reinforcement learning for distributed job-shop scheduling

Effective Multi-User Delay-Constrained Scheduling with Deep Recurrent Reinforcement Learning

Multi-User Delay-Constrained Scheduling With Deep Recurrent Reinforcement Learning

Reinforcement Learning Strategy for A-Team Solving the Resource-Constrained Project Scheduling Problem

Combining Reinforcement Learning Algorithms with Graph Neural Networks to Solve Dynamic Job Shop Scheduling Problems

Flexible robotic cell scheduling with graph neural network based deep reinforcement learning

Petri-net-based dynamic scheduling of flexible manufacturing system via deep reinforcement learning with graph convolutional network

Deep Reinforcement Learning for the Agile Earth Observation Satellite Scheduling Problem

Dynamic Selection of Priority Rules Based on Deep Reinforcement Learning for Rescheduling of RCPSP

Residual Scheduling: A New Reinforcement Learning Approach to Solving Job Shop Scheduling Problem

Deep reinforcement learning for dynamic distributed job shop scheduling problem with transfers