Abstract:PAC-Bayesian algorithms and Gibbs posteriors are gaining popularity due to their robustness against model misspecification even when Bayesian inference is inconsistent. The PAC-Bayesian alpha-posterior is a generalization of the standard Bayes posterior which can be tempered with a parameter alpha to handle inconsistency. Data driven methods for tuning alpha have been proposed but are still few, and are often computationally heavy. Additionally, the adequacy of these methods in cases where we use variational approximations instead of exact alpha-posteriors is not clear. This narrows their usage to simple models and prevents their application to large-scale problems. We hence need fast methods to tune alpha that work with both exact and variational alpha-posteriors. First, we propose two data driven methods for tuning alpha, based on sample-splitting and bootstrapping respectively. Second, we formulate the (exact or variational) posteriors of three popular statistical models, and modify them into alpha-posteriors. For each model, we test our strategies and compare them with standard Bayes and Grunwald's SafeBayes. While bootstrapping achieves mixed results, sample-splitting and SafeBayes perform well on the exact and variational alpha-posteriors we describe, and achieve better results than standard Bayes in misspecified or complex models. Additionally, sample-splitting outperforms SafeBayes in terms of speed. Sample-splitting offers a fast and easy solution to inconsistency and typically performs similarly or better than Bayesian inference. Our results provide hints on the calibration of alpha in PAC-Bayesian and Gibbs posteriors, and may facilitate using these methods in large and complex models.

Practical calibration of the temperature parameter in Gibbs posteriors

On the properties of variational approximations of Gibbs posteriors

Bayesian Model Calibration for Extrapolative Prediction via Gibbs Posteriors

Stratified sampling and bootstrapping for approximate Bayesian computation

posteriordb: Testing, Benchmarking and Developing Bayesian Inference Algorithms

Generalized Posteriors in Approximate Bayesian Computation

The fine print on tempered posteriors

Practical Bayesian Optimization of Machine Learning Algorithms

Reproducible Parameter Inference Using Bagged Posteriors

Insufficient Gibbs sampling

Bayesian Calibration in a multi-output transposition context

Sparsity promoting hybrid solvers for hierarchical Bayesian inverse problems

Using Autodiff to Estimate Posterior Moments, Marginals and Samples

Scalable Approximations of Marginal Posteriors in Variable Selection

Validated Variational Inference via Practical Posterior Error Bounds

Bayesian Adaptive Calibration and Optimal Design

Calibrating general posterior credible regions

Iterative Construction of Gaussian Process Surrogate Models for Bayesian Inference

On Cold Posteriors of Probabilistic Neural Networks: Understanding the Cold Posterior Effect and A New Way to Learn Cold Posteriors with Tight Generalization Guarantees

Iterative Importance Sampling Algorithms for Parameter Estimation

Intuitive joint priors for variance parameters