Abstract:In the context of modern engineering, environmental, and societal concerns, there is an increasing demand for methods able to identify rational management strategies for civil engineering systems, minimizing structural failure risks while optimally planning inspection and maintenance (I&M) processes. Most available methods simplify the I&M decision problem to the component level, often assuming statistical, structural, or cost independence among components, due to the computational complexity associated with global optimization methodologies under joint system-level state descriptions. In this paper, we propose an efficient algorithmic framework for inference and decision-making under uncertainty for engineering systems exposed to deteriorating environments, providing optimal management strategies directly at the system level. In our approach, the decision problem is formulated as a factored partially observable Markov decision process, whose dynamics are encoded in Bayesian network conditional structures. The methodology can handle environments under equal or general, unequal deterioration correlations among components, through Gaussian hierarchical structures and dynamic Bayesian networks, decoupling the originally joint system state space to component networks conditional on shared random variables. In terms of policy optimization, we adopt a deep decentralized multi-agent actor-critic (DDMAC) reinforcement learning approach, in which the policies are approximated by actor neural networks guided by a critic network. By including deterioration dependence in the simulated environment, and by formulating the cost model at the system level, DDMAC policies intrinsically consider the underlying system-effects. This is demonstrated through numerical experiments conducted for both a 9-out-of-10 system and a steel frame under fatigue deterioration. Results demonstrate that DDMAC policies offer substantial benefits when compared to state-of-the-art heuristic approaches. The inherent consideration of system-effects by DDMAC strategies is also interpreted based on the learned policies.

Deep reinforcement learning for intelligent risk optimization of buildings under hazard

Look Before You Leap: Safe Model-Based Reinforcement Learning with Human Intervention

Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings

A probabilistic deep reinforcement learning approach for optimal monitoring of a building adjacent to deep excavation

Methodology for effective and efficient regional seismic retrofit using machine learning and stochastic optimization

Multi-agent deep reinforcement learning based decision support model for resilient community post-hazard recovery

A Transfer Learning Approach to Minimize Reinforcement Learning Risks in Energy Optimization for Smart Buildings

Risk management system and intelligent decision-making for prefabricated building project under deep learning modified teaching-learning-based optimization

Reinforcement Learning with Dynamic Convex Risk Measures

Inference and dynamic decision-making for deteriorating systems with probabilistic dependencies through Bayesian networks and deep reinforcement learning

Resilience-based post disaster recovery optimization for infrastructure system via Deep Reinforcement Learning

Strategic framework for natural disaster risk mitigation using deep learning and cost-benefit analysis

Risk-Sensitive Deep RL: Variance-Constrained Actor-Critic Provably Finds Globally Optimal Policy

A Strategic Framework for Natural Disaster-Induced Cost Risk Analysis and Mitigation: A Two-Stage Approach Using Deep Learning and Cost-Benefit Analysis

Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints

Data‐driven model for seismic assessment, design, and retrofit of structures using explainable artificial intelligence

Explainable deep learning powered building risk assessment model for proactive hurricane response

Uncertainty-Aware Reinforcement Learning for Portfolio Optimization

Risk-Averse Reinforcement Learning via Dynamic Time-Consistent Risk Measures

Optimal Policy for Structure Maintenance: A Deep Reinforcement Learning Framework

Building Retrofit Optimization Considering Future Climate and Decision-Making under Various Mindsets