Abstract:In the context of modern engineering, environmental, and societal concerns, there is an increasing demand for methods able to identify rational management strategies for civil engineering systems, minimizing structural failure risks while optimally planning inspection and maintenance (I&M) processes. Most available methods simplify the I&M decision problem to the component level, often assuming statistical, structural, or cost independence among components, due to the computational complexity associated with global optimization methodologies under joint system-level state descriptions. In this paper, we propose an efficient algorithmic framework for inference and decision-making under uncertainty for engineering systems exposed to deteriorating environments, providing optimal management strategies directly at the system level. In our approach, the decision problem is formulated as a factored partially observable Markov decision process, whose dynamics are encoded in Bayesian network conditional structures. The methodology can handle environments under equal or general, unequal deterioration correlations among components, through Gaussian hierarchical structures and dynamic Bayesian networks, decoupling the originally joint system state space to component networks conditional on shared random variables. In terms of policy optimization, we adopt a deep decentralized multi-agent actor-critic (DDMAC) reinforcement learning approach, in which the policies are approximated by actor neural networks guided by a critic network. By including deterioration dependence in the simulated environment, and by formulating the cost model at the system level, DDMAC policies intrinsically consider the underlying system-effects. This is demonstrated through numerical experiments conducted for both a 9-out-of-10 system and a steel frame under fatigue deterioration. Results demonstrate that DDMAC policies offer substantial benefits when compared to state-of-the-art heuristic approaches. The inherent consideration of system-effects by DDMAC strategies is also interpreted based on the learned policies.

Sequential Monte Carlo-based fidelity selection in dynamic-data-driven adaptive multi-scale simulations

Achieving Robust and Transferable Performance for Conservation‐Based Models of Dynamical Physical Systems

Markov-chain Monte-Carlo Sampling for Optimal Fidelity Determination in Dynamic Decision-Making

Multi-fidelity Sampling for Efficient Simulation-Based Decision Making in Manufacturing Management

Machine-learning-based dynamic-importance sampling for adaptive multiscale simulations

Adaptive Computing for Scale-up Problems

Dynamic Sampling Allocation under Finite Simulation Budget for Feasibility Determination

A sequential Monte Carlo algorithm for data assimilation problems in ecology

Potential of data-driven simulation-based optimization for adaptive scheduling and control of dynamic manufacturing systems

Hierarchical deep learning-based adaptive time stepping scheme for multiscale simulations

Adaptive multilevel subset simulation with selective refinement

Efficient Simulation Sampling Allocation Using Multifidelity Models

Hyperparameter-optimized Multi-fidelity Deep Neural Network model associated with Subset Simulation for structural reliability analysis

A Tutorial on Advanced Dynamic Monte Carlo Methods for Systems with Discrete State Spaces

Inference and dynamic decision-making for deteriorating systems with probabilistic dependencies through Bayesian networks and deep reinforcement learning

Using High-fidelity Time-Domain Simulation Data to Construct Multi-fidelity State Derivative Function Surrogate Models for use in Control and Optimization

A Multiscale Framework for Capturing Oscillation Dynamics of Autonomous Vehicles in Data-Driven Car-Following Models

Adaptive Dynamic Scheduling on Multifunctional Mixed-Criticality Automotive Cyber-Physical Systems.

Towards Improved Uncertainty Quantification of Stochastic Epidemic Models Using Sequential Monte Carlo

Reliable Proactive Adaptation Via Prediction Fusion and Extended Stochastic Model Predictive Control