Abstract:Uncertainty quantification (UQ) is important to machine learning (ML) force fields to assess the level of confidence during prediction, as ML models are not inherently physical and can therefore yield catastrophically incorrect predictions. Established a-posteriori UQ methods, including ensemble methods, the dropout method, the delta method, and various heuristic distance metrics, have limitations such as being computationally challenging for large models due to model re-training. In addition, the uncertainty estimates are often not rigorously calibrated. In this work, we propose combining the distribution-free UQ method, known as conformal prediction (CP), with the distances in the neural network's latent space to estimate the uncertainty of energies predicted by neural network force fields. We evaluate this method (CP+latent) along with other UQ methods on two essential aspects, calibration, and sharpness, and find this method to be both calibrated and sharp under the assumption of independent and identically-distributed (i.i.d.) data. We show that the method is relatively insensitive to hyperparameters selected, and test the limitations of the method when the i.i.d. assumption is violated. Finally, we demonstrate that this method can be readily applied to trained neural network force fields with traditional and graph neural network architectures to obtain estimates of uncertainty with low computational costs on a training dataset of 1 million images to showcase its scalability and portability. Incorporating the CP method with latent distances offers a calibrated, sharp and efficient strategy to estimate the uncertainty of neural network force fields. In addition, the CP approach can also function as a promising strategy for calibrating uncertainty estimated by other approaches.

Ensemble Multi-Quantiles: Adaptively Flexible Distribution Prediction for Uncertainty Quantification

Distributed Bootstrap Simultaneous Inference for High-Dimensional Quantile Regression

Ensemble Deep Learning-Based Non-Crossing Quantile Regression for Nonparametric Probabilistic Forecasting of Wind Power Generation

Regression via Arbitrary Quantile Modeling

Uncertainty separation via ensemble quantile regression

Uncertainty Voting Ensemble for Imbalanced Deep Regression

Deep Ensembles Meets Quantile Regression: Uncertainty-aware Imputation for Time Series

Towards reliable uncertainty quantification via deep ensemble in multi-output regression task

A Distribution-Free Method for Probabilistic Prediction

Quantile Extreme Gradient Boosting for Uncertainty Quantification

Easy Uncertainty Quantification (EasyUQ): Generating Predictive Distributions from Single-Valued Model Output

Towards Reliable Uncertainty Quantification via Deep Ensembles in Multi-output Regression Task

Logit-Based Ensemble Distribution Distillation for Robust Autoregressive Sequence Uncertainties

Deviation Entropy-Based Dynamic Multi-Model Ensemble Interval Prediction Method for Quantifying Uncertainty of Building Cooling Load

Robust and scalable uncertainty estimation with conformal prediction for machine-learned interatomic potentials

Online Prediction of Extreme Conditional Quantiles via B-Spline Interpolation

Uncertainty in Gradient Boosting via Ensembles

Remaining Useful Life Prediction with Uncertainty Quantification Based on Multi-Distribution Fusion Structure

Distribution-free uncertainty quantification for classification under label shift

Distributional reinforcement learning with epistemic and aleatoric uncertainty estimation