Abstract:Neural networks (NNs) are currently changing the computational paradigm on how to combine data with mathematical laws in physics and engineering in a profound way, tackling challenging inverse and ill-posed problems not solvable with traditional methods. However, quantifying errors and uncertainties in NN-based inference is more complicated than in traditional methods. This is because in addition to aleatoric uncertainty associated with noisy data, there is also uncertainty due to limited data, but also due to NN hyperparameters, overparametrization, optimization and sampling errors as well as model misspecification. Although there are some recent works on uncertainty quantification (UQ) in NNs, there is no systematic investigation of suitable methods towards quantifying the total uncertainty effectively and efficiently even for function approximation, and there is even less work on solving partial differential equations and learning operator mappings between infinite-dimensional function spaces using NNs. In this work, we present a comprehensive framework that includes uncertainty modeling, new and existing solution methods, as well as evaluation metrics and post-hoc improvement approaches. To demonstrate the applicability and reliability of our framework, we present an extensive comparative study in which various methods are tested on prototype problems, including problems with mixed input-output data, and stochastic problems in high dimensions. In the Appendix, we include a comprehensive description of all the UQ methods employed. Further, to help facilitate the deployment of UQ in Scientific Machine Learning research and practice, we present and develop in [1] an open-source Python library ( github.com/Crunch-UQ4MI/neuraluq ), termed NeuralUQ, that is accompanied by an educational tutorial and additional computational experiments.

Second-Order Uncertainty Quantification: Variance-Based Measures

Second-Order Uncertainty Quantification: A Distance-Based Approach

Label-wise Aleatoric and Epistemic Uncertainty Quantification

Near-Optimal Variance-Based Uncertainty Relations

On Information-Theoretic Measures of Predictive Uncertainty

Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?

Uncertainty-Based Rejection in Machine Learning: Implications for Model Development and Interpretability

Uncertainty Quantification by Random Measures and Fields

Evidential Uncertainty Quantification: A Variance-Based Perspective

Uncertainty Quantification for Markov Processes via Variational Principles and Functional Inequalities

Quantifying Aleatoric and Epistemic Uncertainty with Proper Scoring Rules

Variational Inference for Uncertainty Quantification: an Analysis of Trade-offs

Quality measures for the evaluation of machine learning architectures on the quantification of epistemic and aleatoric uncertainties in complex dynamical systems

Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons

System-Level Analysis of Module Uncertainty Quantification in the Autonomy Pipeline

Second-Moment/Order Approximations by Kernel Smoothers with Application to Volatility Estimation

Uncertainty quantification in system identification

Random Measures, ANOVA Models and Quantifying Uncertainty in Randomized Controlled Trials

How disentangled are your classification uncertainties?

Uncertainty relations revisited

Uncertainty quantification in metric spaces