Abstract:We consider the problem of Bayesian regression with trustworthy uncertainty quantification. We define that the uncertainty quantification is trustworthy if the ground truth can be captured by intervals dependent on the predictive distributions with a pre-specified probability. Furthermore, we propose, Trust-Bayes, a novel optimization framework for Bayesian meta learning which is cognizant of trustworthy uncertainty quantification without explicit assumptions on the prior model/distribution of the functions. We characterize the lower bounds of the probabilities of the ground truth being captured by the specified intervals and analyze the sample complexity with respect to the feasible probability for trustworthy uncertainty quantification. Monte Carlo simulation of a case study using Gaussian process regression is conducted for verification and comparison with the Meta-prior algorithm.

What problem does this paper attempt to address?

The paper primarily focuses on the issue of achieving reliable uncertainty quantification in Bayesian regression and proposes a new optimization framework—Trust-Bayes—to address this problem. ### Research Background and Motivation Researchers are concerned with ensuring the successful operation and safety of engineering systems (such as autonomous vehicles, unmanned aerial vehicles, etc.) in the face of unknown uncertainties. To enable these systems to operate autonomously with minimal human intervention, it is necessary to reliably learn and quantify these uncertainties. Bayesian learning methods (including Gaussian process regression, Kalman filtering, Bayesian neural networks, etc.) are widely used for such problems because they can infer the target distribution from data. However, in practical applications, the prior distribution and likelihood function may not be accurately specified, which can lead to unreliable uncertainty quantification results. ### Main Contributions 1. **Definition of Trust Quantification**: The paper first defines the concept of "reliable uncertainty quantification," which means that the true value can be captured by the predicted distribution interval with a preset probability. 2. **Trust-Bayes Framework**: To ensure the reliability of uncertainty quantification, the authors propose a new optimization framework called Trust-Bayes. This framework does not require explicit assumptions about the specific form or distribution of the function but instead uses meta-learning to train the prior distribution, making the uncertainty quantification results more reliable. 3. **Theoretical Analysis**: The authors further analyze the lower bound of the probability that the true value is captured and discuss the sample complexity relative to the feasible probability of reliable uncertainty quantification. ### Experimental Validation To validate the effectiveness and necessity of Trust-Bayes, the authors conducted Monte Carlo simulation experiments, using Gaussian process regression as an algorithm instance for case studies. The experimental results show that compared to existing Meta-prior methods, Trust-Bayes not only converges to the required confidence level faster but also better meets the standards of reliable uncertainty quantification in the test set, i.e., the true value is indeed covered by the predicted interval with high probability. ### Conclusion In summary, the Trust-Bayes framework proposed in this paper provides an effective and practical solution to the problem of reliable uncertainty quantification in Bayesian regression. Through experimental validation, this framework significantly improves the reliability of uncertainty quantification, which is particularly important for safety-critical applications that rely on accurate uncertainty estimates.

Bayesian meta learning for trustworthy uncertainty quantification

On Uncertainty Quantification for Near-Bayes Optimal Algorithms

Post-hoc Uncertainty Learning using a Dirichlet Meta-Model

Uncertainty Quantification for Bayesian Optimization

Meta-Uncertainty in Bayesian Model Comparison

Collaborative and Adaptive Bayesian Optimization for bounding variances and probabilities under hybrid uncertainties

Bayesian Model-Agnostic Meta-Learning

Functional Bayesian Neural Networks for Model Uncertainty Quantification

Model calibration via distributionally robust optimization: On the NASA Langley Uncertainty Quantification Challenge

Probabilistic Meta-Representations Of Neural Networks

Bayesian probabilistic propagation of hybrid uncertainties: Estimation of response expectation function, its variable importance and bounds

Bayesian Surrogate Analysis and Uncertainty Propagation

Bayesian Inference Using the Proximal Mapping: Uncertainty Quantification Under Varying Dimensionality

Quantifying Uncertainty in Transdimensional Markov Chain Monte Carlo Using Discrete Markov Models

QuantProb: Generalizing Probabilities along with Predictions for a Pre-trained Classifier

A Meta-heuristic Approach to Estimate and Explain Classifier Uncertainty

Bayesian Modelling in Practice: Using Uncertainty to Improve Trustworthiness in Medical Applications

Bayesian Neural Networks Uncertainty Quantification with Cubature Rules

Bayesian Uncertainty Quantification for Low-Rank Matrix Completion

Uncertainty Quantification for Sparse Deep Learning

A hybrid framework for improving uncertainty quantification in deep learning-based QSAR regression modeling