Abstract:Recent performance breakthroughs in artificial intelligence (AI) and machine learning (ML), especially advances in deep learning, the availability of powerful and easy-to-use ML libraries (e.g., scikit-learn, TensorFlow, PyTorch), and increasing computational power, have led to unprecedented interest in AI/ML among nuclear engineers. For physics-based computational models, verification, validation, and uncertainty quantification (VVUQ) processes have been very widely investigated, and many methodologies have been developed. However, VVUQ of ML models has been relatively less studied, especially in nuclear engineering. This work focuses on uncertainty quantification (UQ) of ML models as a preliminary step of ML VVUQ, more specifically Deep Neural Networks (DNNs) because they are the most widely used supervised ML algorithm for both regression and classification tasks. This work ai3ms at quantifying the prediction or approximation uncertainties of DNNs when they are used as surrogate models for expensive physical models. Three techniques for UQ of DNNs are compared, namely, Monte Carlo Dropout (MCD), Deep Ensembles (DE), and Bayesian Neural Networks (BNNs). Two nuclear engineering examples are used to benchmark these methods: (1) time-dependent fission gas release data using the Bison code and (2) void fraction simulation based on the Boiling Water Reactor Full-size Fine-Mesh Bundle Tests (BFBT) benchmark using the TRACE code. It is found that the three methods typically require different DNN architectures and hyperparameters to optimize their performance. The UQ results also depend on the amount of training data available and the nature of the data. Overall, all three methods can provide reasonable estimations of the approximation uncertainties. The uncertainties are generally smaller when the mean predictions are close to the test data while the BNN methods usually produce larger uncertainties than MCD and DE.

Deep Neural Network Uncertainty Quantification for LArTPC Reconstruction

A Survey on Uncertainty Quantification Methods for Deep Learning

Uncertainty Quantification for Deep Learning

Aleatory-aware deep uncertainty quantification for transfer learning

Robust and scalable uncertainty estimation with conformal prediction for machine-learned interatomic potentials

Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons

Deeply Uncertain: Comparing Methods of Uncertainty Quantification in Deep Learning Algorithms

Uncertainty Quantification in Deep Neural Networks through Statistical Inference on Latent Space

ELUQuant: Event-Level Uncertainty Quantification in Deep Inelastic Scattering

Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priors

A Novel Approach to Uncertainty Quantification in Groundwater Table Modeling by Automated Predictive Deep Learning

Interpretable Uncertainty Quantification in AI for HEP

Uncertainty quantification for noisy inputs-outputs in physics-informed neural networks and neural operators

Quantification of Deep Neural Network Prediction Uncertainties for VVUQ of Machine Learning Models

A review of uncertainty quantification in deep learning: Techniques, applications and challenges

Quantifying Uncertainty in Deep Learning of Radiologic Images

Deep Learning reconstruction with uncertainty estimation for $γ$ photon interaction in fast scintillator detectors

DeepUQ: Assessing the Aleatoric Uncertainties from two Deep Learning Methods

A Survey on Uncertainty Quantification Methods for Deep Neural Networks: An Uncertainty Source Perspective

Aleatory uncertainty quantification based on multi-fidelity deep neural networks

Statistical methods for resolving poor uncertainty quantification in machine learning interatomic potentials