DiffHybrid-UQ: Uncertainty Quantification for Differentiable Hybrid Neural Modeling

Deepak Akhare,Tengfei Luo,Jian-Xun Wang

2023-12-30

Abstract:The hybrid neural differentiable models mark a significant advancement in the field of scientific machine learning. These models, integrating numerical representations of known physics into deep neural networks, offer enhanced predictive capabilities and show great potential for data-driven modeling of complex physical systems. However, a critical and yet unaddressed challenge lies in the quantification of inherent uncertainties stemming from multiple sources. Addressing this gap, we introduce a novel method, DiffHybrid-UQ, for effective and efficient uncertainty propagation and estimation in hybrid neural differentiable models, leveraging the strengths of deep ensemble Bayesian learning and nonlinear transformations. Specifically, our approach effectively discerns and quantifies both aleatoric uncertainties, arising from data noise, and epistemic uncertainties, resulting from model-form discrepancies and data sparsity. This is achieved within a Bayesian model averaging framework, where aleatoric uncertainties are modeled through hybrid neural models. The unscented transformation plays a pivotal role in enabling the flow of these uncertainties through the nonlinear functions within the hybrid model. In contrast, epistemic uncertainties are estimated using an ensemble of stochastic gradient descent (SGD) trajectories. This approach offers a practical approximation to the posterior distribution of both the network parameters and the physical parameters. Notably, the DiffHybrid-UQ framework is designed for simplicity in implementation and high scalability, making it suitable for parallel computing environments. The merits of the proposed method have been demonstrated through problems governed by both ordinary and partial differentiable equations.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to quantify uncertainty in hybrid neural - differential models. Specifically, although existing hybrid neural models can effectively combine known physical laws with deep learning and improve the prediction ability of complex physical systems, there are still challenges in quantifying the inherent uncertainties in these models. These uncertainties mainly come from multiple aspects, such as aleatoric uncertainty caused by data noise and epistemic uncertainty caused by differences in model forms and data sparsity. The paper proposes a new method - DiffHybrid - UQ, which aims to effectively and efficiently propagate and estimate uncertainty by combining the advantages of deep ensemble Bayesian learning and non - linear transformation. This method can not only distinguish and quantify uncertainties from different sources, but also model aleatoric uncertainty under the Bayesian model averaging framework and use the SGD trajectory ensemble to estimate epistemic uncertainty, thereby providing more reliable and comprehensive uncertainty assessment.

DiffHybrid-UQ: Uncertainty Quantification for Differentiable Hybrid Neural Modeling

Uncertainty Quantification of Two-Phase Flow in Porous Media Via the Coupled-TgNN Surrogate Model

Uncertainty Quantification in Neural Differential Equations

NeuralUQ: A Comprehensive Library for Uncertainty Quantification in Neural Differential Equations and Operators

Joint Parameter and Parameterization Inference with Uncertainty Quantification through Differentiable Programming

Surrogate modelling and uncertainty quantification based on multi-fidelity deep neural network

Epistemic Uncertainty Quantification For Pre-trained Neural Network

Aleatory uncertainty quantification based on multi-fidelity deep neural networks

Evaluating Uncertainty Quantification approaches for Neural PDEs in scientific applications

Uncertainty Quantification for Forward and Inverse Problems of PDEs via Latent Global Evolution

A Kernel Framework to Quantify a Model's Local Predictive Uncertainty under Data Distributional Shifts

Uncertainty quantification for noisy inputs-outputs in physics-informed neural networks and neural operators

Uncertainty quantification for deep learning-based schemes for solving high-dimensional backward stochastic differential equations

Uncertainty Quantification for Deep Learning

Leveraging Viscous Hamilton–Jacobi PDEs for Uncertainty Quantification in Scientific Machine Learning

Leveraging viscous Hamilton-Jacobi PDEs for uncertainty quantification in scientific machine learning

A hybrid framework for improving uncertainty quantification in deep learning-based QSAR regression modeling

Robust and scalable uncertainty estimation with conformal prediction for machine-learned interatomic potentials

Uncertainty quantification for noisy inputs–outputs in physics-informed neural networks and neural operators

Total Uncertainty Quantification in Inverse PDE Solutions Obtained with Reduced-Order Deep Learning Surrogate Models

Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial