Abstract:In the context of modern engineering, environmental, and societal concerns, there is an increasing demand for methods able to identify rational management strategies for civil engineering systems, minimizing structural failure risks while optimally planning inspection and maintenance (I&M) processes. Most available methods simplify the I&M decision problem to the component level, often assuming statistical, structural, or cost independence among components, due to the computational complexity associated with global optimization methodologies under joint system-level state descriptions. In this paper, we propose an efficient algorithmic framework for inference and decision-making under uncertainty for engineering systems exposed to deteriorating environments, providing optimal management strategies directly at the system level. In our approach, the decision problem is formulated as a factored partially observable Markov decision process, whose dynamics are encoded in Bayesian network conditional structures. The methodology can handle environments under equal or general, unequal deterioration correlations among components, through Gaussian hierarchical structures and dynamic Bayesian networks, decoupling the originally joint system state space to component networks conditional on shared random variables. In terms of policy optimization, we adopt a deep decentralized multi-agent actor-critic (DDMAC) reinforcement learning approach, in which the policies are approximated by actor neural networks guided by a critic network. By including deterioration dependence in the simulated environment, and by formulating the cost model at the system level, DDMAC policies intrinsically consider the underlying system-effects. This is demonstrated through numerical experiments conducted for both a 9-out-of-10 system and a steel frame under fatigue deterioration. Results demonstrate that DDMAC policies offer substantial benefits when compared to state-of-the-art heuristic approaches. The inherent consideration of system-effects by DDMAC strategies is also interpreted based on the learned policies.

Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints

Real-Time Integrated Learning and Decision-Making for Asset Networks

A Deep Reinforcement Learning Approach for Maintenance Planning of Multi-Component Systems with Complex Structure

Multi-agent deep reinforcement learning with centralized training and decentralized execution for transportation infrastructure management

An investigation of belief-free DRL and MCTS for inspection and maintenance planning

A Maintenance Planning Framework using Online and Offline Deep Reinforcement Learning

Inference and dynamic decision-making for deteriorating systems with probabilistic dependencies through Bayesian networks and deep reinforcement learning

Optimal Inspection and Maintenance Planning for Deteriorating Structural Components through Dynamic Bayesian Networks and Markov Decision Processes

A deep reinforcement learning approach for real-time sensor-driven decision making and predictive analytics

A deep reinforcement learning framework for life-cycle maintenance planning of regional deteriorating bridges using inspection data

Dynamic Selective Maintenance Optimization for Multi-State Systems over a Finite Horizon: A Deep Reinforcement Learning Approach.

Dynamic Inspection and Maintenance Scheduling for Multi-State Systems under Time-Varying Demand: Proximal Policy Optimization

Deep Reinforce Learning for Joint Optimization of Condition-Based Maintenance and Spare Ordering.

Scalable Policies for the Dynamic Traveling Multi-Maintainer Problem with Alerts

Optimal Policy for Structure Maintenance: A Deep Reinforcement Learning Framework

Multi-agent deep reinforcement learning based Predictive Maintenance on parallel machines

An Offline Deep Reinforcement Learning for Maintenance Decision-Making

Guided Probabilistic Reinforcement Learning for Sampling-Efficient Maintenance Scheduling of Multi-Component System

Smart robot-enabled remaining useful life prediction and maintenance optimization for complex structures using artificial intelligence and machine learning

A Dynamic Maintenance Policy for Degradation System by State Monitoring and Deep Reinforcement Learning

Adaptive Disassembly Sequence Planning for VR Maintenance Training Via Deep Reinforcement Learning