Abstract:Neural networks (NNs) are currently changing the computational paradigm on how to combine data with mathematical laws in physics and engineering in a profound way, tackling challenging inverse and ill-posed problems not solvable with traditional methods. However, quantifying errors and uncertainties in NN-based inference is more complicated than in traditional methods. This is because in addition to aleatoric uncertainty associated with noisy data, there is also uncertainty due to limited data, but also due to NN hyperparameters, overparametrization, optimization and sampling errors as well as model misspecification. Although there are some recent works on uncertainty quantification (UQ) in NNs, there is no systematic investigation of suitable methods towards quantifying the total uncertainty effectively and efficiently even for function approximation, and there is even less work on solving partial differential equations and learning operator mappings between infinite-dimensional function spaces using NNs. In this work, we present a comprehensive framework that includes uncertainty modeling, new and existing solution methods, as well as evaluation metrics and post-hoc improvement approaches. To demonstrate the applicability and reliability of our framework, we present an extensive comparative study in which various methods are tested on prototype problems, including problems with mixed input-output data, and stochastic problems in high dimensions. In the Appendix, we include a comprehensive description of all the UQ methods employed. Further, to help facilitate the deployment of UQ in Scientific Machine Learning research and practice, we present and develop in [1] an open-source Python library ( github.com/Crunch-UQ4MI/neuraluq ), termed NeuralUQ, that is accompanied by an educational tutorial and additional computational experiments.

Metrics for Benchmarking and Uncertainty Quantification: Quality, Applicability, and a Path to Best Practices for Machine Learning in Chemistry

Characterizing Uncertainty in Machine Learning for Chemistry

Methods for comparing uncertainty quantifications for material property predictions

Validation of uncertainty quantification metrics: a primer based on the consistency and adaptivity concepts

UNIQUE: A Framework for Uncertainty Quantification Benchmarking

Uncertainty Quantification and Propagation in Atomistic Machine Learning

Statistical methods for resolving poor uncertainty quantification in machine learning interatomic potentials

Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons

Uncertainty Quantification Metrics for Deep Regression

Error Assessment of Computational Models in Chemistry

Chemoinformatic regression methods and their applicability domain

Navigating with chemometrics and machine learning in chemistry

Uncertainty reduction and quantification in computational thermodynamics

A Structured Review of Literature on Uncertainty in Machine Learning & Deep Learning

New Metrics for Validation of Data-Driven Random Process Models in Uncertainty Quantification

Comparative Study of Uncertainty Quantification Metrics via a Stochastic Method of Model Validation

Uncertainty Quantification Using Neural Networks for Molecular Property Prediction

Understanding metric-related pitfalls in image analysis validation

Toward a Better Understanding of Model Validation Metrics

Machine Learning for Chemistry: Basics and Applications