Abstract:Data-driven surrogate models have shown immense potential as quick, inexpensive approximations to complex numerical and experimental modelling tasks. However, most surrogate models of physical systems do not quantify their uncertainty, rendering their predictions unreliable, requiring further validation. Though Bayesian approximations offer some solace in estimating the error associated with these models, they cannot provide guarantees, and the quality of their inferences depends on the availability of prior information and good approximations to posteriors for complex problems. This is particularly pertinent to multi-variable or spatio-temporal problems. Our work constructs and formalises a conformal prediction framework that satisfies marginal coverage for spatio-temporal predictions in a model-agnostic manner, requiring near-zero computational costs. We provide an extensive empirical study of the application of the framework to ascertain valid error bars that provide guaranteed coverage across the surrogate model's domain of operation. The application scope of our work extends across a large range of spatio-temporal models, from solving partial differential equations to weather forecasting. Through the applications, the paper looks at providing statistically valid error bars for deterministic models, as well as crafting guarantees to the error bars of probabilistic models. Our conformal prediction formalisation provides guaranteed coverage of the surrogate model, regardless of model architecture, and its training regime and is unbothered by the curse of dimensionality.

Cubature-based uncertainty estimation for nonlinear regression models

Nonlinear Recursive Minimum Model Error Estimation

Uncertainty Quantification Metrics for Deep Regression

Evaluating and Calibrating Uncertainty Prediction in Regression Tasks

Accurate Uncertainties for Deep Learning Using Calibrated Regression

Novel Model Calibration Method Via Non-Probabilistic Interval Characterization and Bayesian Theory

Worst-case parameter estimation with bounded model uncertainties

Two-stage Nested Optimization-Based Uncertainty Propagation Method for Model Calibration.

Parameter uncertainties for imperfect surrogate models in the low-noise regime

A Novel Regression Loss for Non-Parametric Uncertainty Optimization

Towards calibrated and scalable uncertainty representations for neural networks

A prediction rigidity formalism for low-cost uncertainties in trained neural networks

Quantifying Aleatoric and Epistemic Dynamics Uncertainty via Local Conformal Calibration

Uncertainty Quantification Using Polynomial Chaos Expansion with Points of Monomial Cubature Rules

A collaborative model calibration framework under uncertainty considering parameter distribution

Estimating Unknown Parameters and Disturbance Term in Uncertain Regression Models by the Principle of Least Squares

Single-Model Uncertainties for Deep Learning

Semi-Parametric Functional Calibration Using Uncertainty Quantification Based Decision Support

An Uncertainty Quantification Method for Inexact Simulation Models

Bayesian Neural Networks Uncertainty Quantification with Cubature Rules

Uncertainty Quantification of Surrogate Models using Conformal Prediction