Abstract:Abstract Maintenance planning aims to improve the reliability of assets, prevent the occurrence of asset failures, and reduce maintenance costs associated with downtime of assets and maintenance resources (such as spare parts and workforce). Thus, effective maintenance planning is instrumental in ensuring high asset availability with the minimum cost. Nevertheless, to find such optimal planning is a nontrivial task due to the (i) complex and usually nonlinear inter-relationship between different planning decisions (e.g., inventory level and workforce capacity), and (ii) stochastic nature of the system (e.g., random failures of parts installed in assets). To alleviate these challenges, we study a joint maintenance planning problem by considering several decisions simultaneously, including workforce planning, workforce training, and spare parts inventory management. We develop a hybrid solution algorithm ( $$\mathcal {DRLSA}$$ DRLSA ) that is a combination of Double Deep Q-Network based Deep Reinforcement Learning (DRL) and Simulated Annealing (SA) algorithms. In each episode of the proposed algorithm, the best solution found by DRL is delivered to SA to be used as an initial solution, and the best solution of SA is delivered to DRL to be used as the initial state. Different from the traditional SA algorithms where neighborhood structures are selected only randomly, the DRL part of $$\mathcal {DRLSA}$$ DRLSA learns to choose the best neighborhood structure to use based on experience gained from previous episodes. We compare the performance of the proposed solution algorithm with several well-known meta-heuristic algorithms, including Simulated Annealing, Genetic Algorithm (GA), and Variable Neighborhood Search (VNS). Further, we also develop a Machine Learning (ML) algorithm (i.e., K-Median) as another benchmark in which different properties of spare parts (e.g., failure rates, holding costs, and repair rates) are used as clustering features for the ML algorithm. Our study reveals that the $$\mathcal {DRLSA}$$ DRLSA finds the optimal solutions for relatively small-size instances, and it has the potential to outperform traditional meta-heuristic and ML algorithms.

Aircraft Line Maintenance Scheduling Using Simulation and Reinforcement Learning

Reinforcement learning-driven maintenance strategy: A novel solution for long-term aircraft maintenance decision optimization

A deep reinforcement learning assisted simulated annealing algorithm for a maintenance planning problem

Guided Probabilistic Reinforcement Learning for Sampling-Efficient Maintenance Scheduling of Multi-Component System

A Survey on Reinforcement Learning in Aviation Applications

Deep Reinforcement Learning for Real-Time Ground Delay Program Revision and Corresponding Flight Delay Assignments

Predictive Maintenance Scheduling for Aircraft Engines Based On Remaining Useful Life Prediction

Autonomous Airline Revenue Management: A Deep Reinforcement Learning Approach to Seat Inventory Control and Overbooking

Intelligent Scheduling of Discrete Automated Production Line Via Deep Reinforcement Learning

A deep reinforcement learning approach for dynamic task scheduling of flight tests

A Reinforcement Learning-Artificial Bee Colony algorithm for Flexible Job-shop Scheduling Problem with Lot Streaming

An Improved Optimization Algorithm for Aeronautical Maintenance and Repair Task Scheduling Problem

Reinforcement Learning based Condition-oriented Maintenance Scheduling for Flow Line Systems

A Review of Reinforcement Learning Based Intelligent Optimization for Manufacturing Scheduling

Optimizing Job Shop Scheduling in the Furniture Industry: A Reinforcement Learning Approach Considering Machine Setup, Batch Variability, and Intralogistics

Collaborative-sequential optimization for aero-engine maintenance based on multi-agent reinforcement learning

A Two-stage Framework and Reinforcement Learning-based Optimization Algorithms for Complex Scheduling Problems

A Graph-based Adversarial Imitation Learning Framework for Reliable & Realtime Fleet Scheduling in Urban Air Mobility

Imperfect Maintenance Optimization of Multi-State Rolling Stocks Based on Deep Reinforcement Learning

Autonomous air traffic separation assurance through machine learning

A Deep Reinforcement Learning Approach for Optimal Scheduling of Heavy-haul Railway