Abstract:This paper studies how to model and resolve the dynamic maintenance problem for the continuous degrading system. The main objective is to realize the maintenance cost optimization and determine the optimal maintenance action in the real-time degradation system. First, a deterioration model based on the Wiener process with Brownian motion is described to simulate the deterioration system and estimate the deterioration level. Second, a data-processing method is applied to discretize the continuous degradation into the discrete degradation state. Thus, a discrete-time finite Markov decision process (MDP) model is established to describe the maintenance decision processes. Third, a dynamic maintenance decision framework is constructed by periodic status monitoring and a customized deep reinforcement learning method. The developed decision framework determines the maintenance action by utilizing the specific degradation state at each inspection stage. Fourth, a maintenance decision agent is developed by employing a customized proximal policy optimization with an entropy regularization approach to solve the dynamic maintenance problem, in which the deep actor- critic networks, advantage function estimation method, truncated surrogate approach, and entropy regularization approach are integrated to facilitate the policy optimization processes, and the customized maintenance decision approach is also verified by simulation analysis. To illustrate the effectiveness, comparison studies are conducted and the results demonstrate that: (i) the cumulative maintenance costs are decreased and approaching the approximated equilibrium state by running the proposed maintenance decision algorithm; (ii) the proposed customized deep reinforcement learning approach for the maintenance decision problem can be implemented easily, and the efficiency and stability can be enhanced distinctively; (iii) the proposed maintenance framework can provide a useful decision tool for engineering manager.

A Framework for Modeling and Optimizing Maintenance in Systems Considering Epistemic Uncertainty and Degradation Dependence Based on PDMPs.

Maintenance policy optimization for multi-component systems considering dynamic importance of components

Piecewise deterministic Markov process for condition-based imperfect maintenance models

Modelling and Optimizing Sequential Imperfect Preventive Maintenance

Optimum Life-Cycle Maintenance Strategies of Deteriorating Highway Bridges Subject to Seismic Hazard by a Hybrid Markov Decision Process Model

Maintenance optimisation for systems with multi-dimensional degradation and imperfect inspections

A Prognosis-Centered Intelligent Maintenance Optimization Framework Under Uncertain Failure Threshold

A Dynamic Maintenance Policy for Degradation System by State Monitoring and Deep Reinforcement Learning

Optimization of preventive maintenance for a multi-state degraded system by monitoring component performance

Generalized Condition-Based Maintenance Optimization for Multi-Component Systems Considering Stochastic Dependency and Imperfect Maintenance.

Condition-based maintenance policy for systems under dynamic environment

Optimal Inspection and Maintenance Planning for Deteriorating Structural Components through Dynamic Bayesian Networks and Markov Decision Processes

A risk-aware maintenance model based on a constrained Markov decision process

A condition-based maintenance policy for a two-component balanced system with dependent degradation processes

Optimal Maintenance Policy for Multi-Component Systems under Markovian Environment Changes.

Real-time prediction of remaining useful life and preventive opportunistic maintenance strategy for multi-component systems considering stochastic dependence

Condition-based maintenance assessment for a deteriorating system considering stochastic failure dependence

A two-dimensional maintenance optimization framework balancing hazard risk and energy consumption rates

Optimal rearrangement and preventive maintenance policies for heterogeneous balanced systems with three failure modes

Optimal maintenance strategy for multi-state systems with single maintenance capacity and arbitrarily distributed maintenance time

Maintaining Systems with Dependent Failure Modes and Resource Constraints