Abstract:Abstract Maintenance planning aims to improve the reliability of assets, prevent the occurrence of asset failures, and reduce maintenance costs associated with downtime of assets and maintenance resources (such as spare parts and workforce). Thus, effective maintenance planning is instrumental in ensuring high asset availability with the minimum cost. Nevertheless, to find such optimal planning is a nontrivial task due to the (i) complex and usually nonlinear inter-relationship between different planning decisions (e.g., inventory level and workforce capacity), and (ii) stochastic nature of the system (e.g., random failures of parts installed in assets). To alleviate these challenges, we study a joint maintenance planning problem by considering several decisions simultaneously, including workforce planning, workforce training, and spare parts inventory management. We develop a hybrid solution algorithm ( $$\mathcal {DRLSA}$$ DRLSA ) that is a combination of Double Deep Q-Network based Deep Reinforcement Learning (DRL) and Simulated Annealing (SA) algorithms. In each episode of the proposed algorithm, the best solution found by DRL is delivered to SA to be used as an initial solution, and the best solution of SA is delivered to DRL to be used as the initial state. Different from the traditional SA algorithms where neighborhood structures are selected only randomly, the DRL part of $$\mathcal {DRLSA}$$ DRLSA learns to choose the best neighborhood structure to use based on experience gained from previous episodes. We compare the performance of the proposed solution algorithm with several well-known meta-heuristic algorithms, including Simulated Annealing, Genetic Algorithm (GA), and Variable Neighborhood Search (VNS). Further, we also develop a Machine Learning (ML) algorithm (i.e., K-Median) as another benchmark in which different properties of spare parts (e.g., failure rates, holding costs, and repair rates) are used as clustering features for the ML algorithm. Our study reveals that the $$\mathcal {DRLSA}$$ DRLSA finds the optimal solutions for relatively small-size instances, and it has the potential to outperform traditional meta-heuristic and ML algorithms.

Deep reinforcement learning with combinatorial actions spaces: An application to prescriptive maintenance

Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints

An Offline Deep Reinforcement Learning for Maintenance Decision-Making

A deep reinforcement learning approach for real-time sensor-driven decision making and predictive analytics

Real-Time Integrated Learning and Decision-Making for Asset Networks

Multi-agent deep reinforcement learning based Predictive Maintenance on parallel machines

Adaptive Disassembly Sequence Planning for VR Maintenance Training Via Deep Reinforcement Learning

TranDRL: A Transformer-Driven Deep Reinforcement Learning Enabled Prescriptive Maintenance Framework

Reinforcement learning-driven maintenance strategy: A novel solution for long-term aircraft maintenance decision optimization

Reinforcement and Deep Reinforcement Learning-based Solutions for Machine Maintenance Planning, Scheduling Policies, and Optimization

Deep Reinforcement Learning for Dynamic Opportunistic Maintenance of Multi-Component Systems With Load Sharing

Counterfactual-attention multi-agent reinforcement learning for joint condition-based maintenance and production scheduling

A Dynamic Maintenance Policy for Degradation System by State Monitoring and Deep Reinforcement Learning

Synergetic-informed deep reinforcement learning for sustainable management of transportation networks with large action spaces

Reinforcement learning for predictive maintenance: a systematic technical review

A deep reinforcement learning assisted simulated annealing algorithm for a maintenance planning problem

Towards Solving Industrial Sequential Decision-making Tasks under Near-predictable Dynamics via Reinforcement Learning: an Implicit Corrective Value Estimation Approach

Deep Multi-Objective Reinforcement Learning for Utility-Based Infrastructural Maintenance Optimization

A Maintenance Planning Framework using Online and Offline Deep Reinforcement Learning

Maintenance optimisation of multi-unit balanced systems using deep reinforcement learning

Deep-Reinforcement-Learning-Based Predictive Maintenance Model for Effective Resource Management in Industrial IoT