Abstract:This paper studies how to model and resolve the dynamic maintenance problem for the continuous degrading system. The main objective is to realize the maintenance cost optimization and determine the optimal maintenance action in the real-time degradation system. First, a deterioration model based on the Wiener process with Brownian motion is described to simulate the deterioration system and estimate the deterioration level. Second, a data-processing method is applied to discretize the continuous degradation into the discrete degradation state. Thus, a discrete-time finite Markov decision process (MDP) model is established to describe the maintenance decision processes. Third, a dynamic maintenance decision framework is constructed by periodic status monitoring and a customized deep reinforcement learning method. The developed decision framework determines the maintenance action by utilizing the specific degradation state at each inspection stage. Fourth, a maintenance decision agent is developed by employing a customized proximal policy optimization with an entropy regularization approach to solve the dynamic maintenance problem, in which the deep actor- critic networks, advantage function estimation method, truncated surrogate approach, and entropy regularization approach are integrated to facilitate the policy optimization processes, and the customized maintenance decision approach is also verified by simulation analysis. To illustrate the effectiveness, comparison studies are conducted and the results demonstrate that: (i) the cumulative maintenance costs are decreased and approaching the approximated equilibrium state by running the proposed maintenance decision algorithm; (ii) the proposed customized deep reinforcement learning approach for the maintenance decision problem can be implemented easily, and the efficiency and stability can be enhanced distinctively; (iii) the proposed maintenance framework can provide a useful decision tool for engineering manager.

A Dynamic Maintenance Policy for Degradation System by State Monitoring and Deep Reinforcement Learning

Condition-based maintenance policy for systems under dynamic environment

Adaptive Residual-Based Maintenance Policy For A Deteriorating System In Dynamic Environment

An Optimal Condition-Based Replacement Method for Systems with Observed Degradation Signals

A Deep Reinforcement Learning Approach to Dynamic Loading Strategy of Repairable Multistate Systems

Dynamic Loading and Condition-Based Maintenance Policies for Multi-State Systems with Periodic Inspection

A Dynamic Maintenance Strategy for Prognostics and Health Management of Degrading Systems: Application in Locomotive Wheel-sets.

An Optimal Maintenance Strategy for Multi-State Deterioration Systems Based on a Semi-Markov Decision Process Coupled with Simulation Technique

Dynamic Selective Maintenance Optimization for Multi-State Systems over a Finite Horizon: A Deep Reinforcement Learning Approach.

Maintenance policy for a system with a weighted linear combination of degradation processes

A Dynamic Prescriptive Maintenance Model Considering System Aging and Degradation

Condition-based Maintenance for Multi-state Systems with Prognostic and Deep Reinforcement Learning

A Finite-Time Maintenance Policy of Deteriorating Systems with Multiple Dependent Performance Characteristics

Availability and optimal maintenance policy for systems degrading in dynamic environments

Cost-Effective Updated Sequential Predictive Maintenance Policy for Continuously Monitored Degrading Systems

An Adaptive Maintenance Policy With Nonlinear Degradation Modeling Based On Prognostic Information

A Framework for Modeling and Optimizing Maintenance in Systems Considering Epistemic Uncertainty and Degradation Dependence Based on PDMPs.

Real-Time Integrated Learning and Decision-Making for Asset Networks

Condition-Based Maintenance Policy for Systems with a Non-Homogeneous Degradation Process.

A condition-based maintenance policy for a two-component balanced system with dependent degradation processes

An Optimal Opportunistic Maintenance Planning Integrating Discrete- and Continuous-State Information