Abstract:Recent performance breakthroughs in artificial intelligence (AI) and machine learning (ML), especially advances in deep learning, the availability of powerful and easy-to-use ML libraries (e.g., scikit-learn, TensorFlow, PyTorch), and increasing computational power, have led to unprecedented interest in AI/ML among nuclear engineers. For physics-based computational models, verification, validation, and uncertainty quantification (VVUQ) processes have been very widely investigated, and many methodologies have been developed. However, VVUQ of ML models has been relatively less studied, especially in nuclear engineering. This work focuses on uncertainty quantification (UQ) of ML models as a preliminary step of ML VVUQ, more specifically Deep Neural Networks (DNNs) because they are the most widely used supervised ML algorithm for both regression and classification tasks. This work ai3ms at quantifying the prediction or approximation uncertainties of DNNs when they are used as surrogate models for expensive physical models. Three techniques for UQ of DNNs are compared, namely, Monte Carlo Dropout (MCD), Deep Ensembles (DE), and Bayesian Neural Networks (BNNs). Two nuclear engineering examples are used to benchmark these methods: (1) time-dependent fission gas release data using the Bison code and (2) void fraction simulation based on the Boiling Water Reactor Full-size Fine-Mesh Bundle Tests (BFBT) benchmark using the TRACE code. It is found that the three methods typically require different DNN architectures and hyperparameters to optimize their performance. The UQ results also depend on the amount of training data available and the nature of the data. Overall, all three methods can provide reasonable estimations of the approximation uncertainties. The uncertainties are generally smaller when the mean predictions are close to the test data while the BNN methods usually produce larger uncertainties than MCD and DE.

QUTE: Quantifying Uncertainty in TinyML with Early-exit-assisted ensembles for model-monitoring

Pse: Mixed Quantization Framework of Neural Networks for Efficient Deployment

An Empirical Study of Low Precision Quantization for TinyML

T-RECX: Tiny-Resource Efficient Convolutional neural networks with early-eXit

Tiny Deep Ensemble: Uncertainty Estimation in Edge AI Accelerators via Ensembling Normalization Layers with Shared Weights

Fast Uncertainty Quantification for Deep Object Pose Estimation

UNIQUE: A Framework for Uncertainty Quantification Benchmarking

Predicting Probabilities of Error to Combine Quantization and Early Exiting: QuEE

Quantification of Deep Neural Network Prediction Uncertainties for VVUQ of Machine Learning Models

Quantization-Guided Training for Compact TinyML Models

QEFT: Quantization for Efficient Fine-Tuning of LLMs

DiffHybrid-UQ: Uncertainty Quantification for Differentiable Hybrid Neural Modeling

Uncertainty Quantification in Table Structure Recognition

DeepUQ: Assessing the Aleatoric Uncertainties from two Deep Learning Methods

Quality measures for the evaluation of machine learning architectures on the quantification of epistemic and aleatoric uncertainties in complex dynamical systems

Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs

Single-Model Uncertainties for Deep Learning

MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantization

Single-model uncertainty quantification in neural network potentials does not consistently outperform model ensembles

QET: Enhancing Quantized LLM Parameters and KV cache Compression through Element Substitution and Residual Clustering

Leveraging Automated Mixed-Low-Precision Quantization for tiny edge microcontrollers