Abstract:During the service life, inspections and repairs should be applied timely to maintain the reliability level of deteriorating structures. Condition-based maintenance (CBM) is an effective maintenance policy to reduce the life cycle cost. When the number of inspections does not change regardless of the performance, the CBM is categorize as fixed inspection (FI), otherwise, the inspection policy is denoted as dynamic inspection (DI). Compared with FI policy, DI policy performs the inspections based on the actual state and can avoid the unnecessary or insufficient inspections. Reinforcement learning is an effective and advanced decision-making tool and provides a useful method to optimize DI policy. Meanwhile, reinforcement learning has two methods (model free and model based) distinguished by the interaction method of environment. Comparison of two methods can help select an appropriate method to derive DI policy. Here, model based dynamic inspection (MBDI) and model free dynamic inspection (MFDI) are investigated for their performances in integrity management of fatigue structures. A fatigue details of ship structure is applied to illustrate the proposed framework and comparison study between DI and FI is performed. Results show that dynamic inspections can effectively reduce the expected life cycle costs. Furthermore, MFDI has a better performance than MBDI under different deteriorating rate and cost conditions.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to optimize the condition - based maintenance (CBM) strategy for fatigue - sensitive structures, especially to reduce the life - cycle cost through dynamic inspection (DI). Specifically, the author compares two different reinforcement learning methods - model - based reinforcement learning (MBRL) and model - free reinforcement learning (MFRL) - to determine which method can more effectively optimize the dynamic inspection policy under different degradation rates and cost conditions. ### Core Problems of the Paper 1. **Optimizing the Condition - Based Maintenance Strategy** - Traditional fixed - inspection (FI) strategies perform the same number of inspections in all cases, which may lead to unnecessary or insufficient inspections. - Dynamic inspection (DI) strategies adjust the inspection frequency according to the actual structure state, thereby avoiding unnecessary or insufficient inspections and improving the adaptability and performance of life - cycle management. 2. **Applying Reinforcement Learning to Optimize DI Strategies** - Reinforcement learning is an effective decision - making tool that can be used to optimize DI strategies. - Compare the performance of model - based reinforcement learning (MBRL) and model - free reinforcement learning (MFRL) in optimizing DI strategies. ### Specific Objectives - **Model Building and Optimization** - Use the Markov decision process (MDP) model and value - iteration algorithm to optimize the model - based dynamic inspection (MBDI) strategy. - Use the First - visit Monte Carlo method to optimize the model - free dynamic inspection (MFDI) strategy. - **Performance Comparison** - Compare the performance of MBDI and MFDI strategies under different degradation rates and cost conditions. - Evaluate the effectiveness of different strategies by calculating the expected life - cycle cost. ### Main Contributions - Propose a method of using reinforcement learning to optimize the dynamic inspection strategy for fatigue - sensitive structures. - Compare the advantages and disadvantages of MBRL and MFRL in optimizing DI strategies, providing a basis for selecting an appropriate reinforcement learning framework. - Verify the effectiveness of the dynamic inspection strategy in reducing life - cycle costs through examples. ### Key Formulas 1. **Fatigue Crack Growth Model** \[ \frac{da}{dN} = C(\Delta K)^M \] where \(a\) is the crack length, \(N\) is the number of loading cycles, \(C\) and \(M\) are material parameters, and \(\Delta K\) is the stress intensity factor. 2. **Fatigue Crack Growth Prediction** \[ a(t) = \begin{cases} \left[a_0^{1-\frac{M}{2}}+(1 - \frac{M}{2}) C S_r^M G^M \pi^{\frac{M}{2}} N_{av} t\right]^{\frac{2}{2 - M}}, & M\neq 2 \\ a_0 \exp(C S_r^M G^M \pi^{\frac{M}{2}} N_{av} t), & M = 2 \end{cases} \] 3. **Life - Cycle Cost Analysis** \[ C_{LTC}=\sum_i \frac{C_I}{(1 + r)^{t_i}}+\sum_j \frac{C_R}{(1 + r)^{t_j}}+\sum_k \frac{C_F}{(1 + r)^{t_k}} \] where \(C_I\) is the inspection cost, \(C_R\) is the repair cost, \(C_F\) is the failure cost, and \(r\) is the discount rate. 4. **Optimal Value Function** \[ V_{\pi^*}(s, t_s)=\max_{a\in A}\left[R(s, a, t_s)+\frac{1}{1 + r}\sum_{s'} p(s'|s, a) V_{\pi^*}(s', t_s + t_{ne})\right] \] Through these formulas...

Optimum condition-based maintenance policy with dynamic inspections based on reinforcement learning

A Condition-Based Replacement and Spare Provisioning Policy for Deteriorating Systems with Uncertain Deterioration to Failure.

Deep reinforcement learning for cost-optimal condition-based maintenance policy of offshore wind turbine components

Probabilistic investigations into the value of information: A comparison of condition-based and time-based maintenance strategies

Condition-based maintenance policy for systems under dynamic environment

Life-cycle multi-attribute decision making of RC structures considering sustainability

Optimal maintenance over a finite time horizon for a system under imperfect inspection and dynamic working environment

A Dynamic Maintenance Policy for Degradation System by State Monitoring and Deep Reinforcement Learning

Inspection-Replacement Policy of System under Predictive Maintenance

SHM-informed life-cycle intelligent maintenance of fatigue-sensitive detail using Bayesian forecasting and Markov decision process

Optimal Policy for Structure Maintenance: A Deep Reinforcement Learning Framework

Generalized Condition-Based Maintenance Optimization for Multi-Component Systems Considering Stochastic Dependency and Imperfect Maintenance.

A Prognosis-Centered Intelligent Maintenance Optimization Framework Under Uncertain Failure Threshold

Optimal Inspection and Maintenance Planning for Deteriorating Structural Components through Dynamic Bayesian Networks and Markov Decision Processes

On Fault Diagnosis and Inspection Policy for Deteriorating System

Dynamic Selective Maintenance Optimization for Multi-State Systems over a Finite Horizon: A Deep Reinforcement Learning Approach.

A Finite-Horizon Condition-Based Maintenance Policy for a Two-Unit System with Dependent Degradation Processes.

Maintenance strategy of highway bridges based on life cycle function

Maintenance optimisation for systems with multi-dimensional degradation and imperfect inspections

Joint optimization of inspection and condition-based maintenance for a deteriorating product under extended warranty

A Condition-Based Maintenance Policy for Intelligent Monitored System