Abstract:An important feature of Bayesian statistics is the opportunity to do sequential inference: the posterior distribution obtained after seeing a dataset can be used as prior for a second inference. However, when Monte Carlo sampling methods are used for inference, we only have a set of samples from the posterior distribution. To do sequential inference, we then either have to evaluate the second posterior at only these locations and reweight the samples accordingly, or we can estimate a functional description of the posterior probability distribution from the samples and use that as prior for the second inference. Here, we investigated to what extent we can obtain an accurate joint posterior from two datasets if the inference is done sequentially rather than jointly, under the condition that each inference step is done using Monte Carlo sampling. To test this, we evaluated the accuracy of kernel density estimates, Gaussian mixtures, mixtures of factor analyzers, vine copulas and Gaussian processes in approximating posterior distributions, and then tested whether these approximations can be used in sequential inference. In low dimensionality, Gaussian processes are more accurate, whereas in higher dimensionality Gaussian mixtures, mixtures of factor analyzers or vine copulas perform better. In our test cases of sequential inference, using posterior approximations gives more accurate results than direct sample reweighting, but joint inference is still preferable over sequential inference whenever possible. Since the performance is case-specific, we provide an R package <em>mvdens</em> with a unified interface for the density approximation methods.

Monte Carlo Approximation of Bayes Factors via Mixing with Surrogate Distributions

Bayesian estimation and prediction for certain mixtures

Hamiltonian Monte Carlo Acceleration Using Surrogate Functions with Random Bases

Stratified sampling and bootstrapping for approximate Bayesian computation

Accelerating sequential Monte Carlo with surrogate likelihoods

Graph-accelerated Markov Chain Monte Carlo using Approximate Samples

Approximating multivariate posterior distribution functions from Monte Carlo samples for sequential Bayesian inference

Merging MCMC Subposteriors through Gaussian-Process Approximations

A generalized Multiple-try Metropolis version of the Reversible Jump algorithm

Non-Log-Concave and Nonsmooth Sampling via Langevin Monte Carlo Algorithms

Model selection in finite mixture of regression models: a Bayesian approach with innovative weighted g priors and reversible jump Markov chain Monte Carlo implementation (vol 85, pg 2456, 2015)

Laplacian Smoothing Stochastic Gradient Markov Chain Monte Carlo

Bayesian Posterior Sampling via Stochastic Gradient Fisher Scoring

Unbiased Kinetic Langevin Monte Carlo with Inexact Gradients

Accelerated Markov Chain Monte Carlo Using Adaptive Weighting Scheme

Parallelized Midpoint Randomization for Langevin Monte Carlo

Efficient stochastic optimisation by unadjusted Langevin Monte Carlo. Application to maximum marginal likelihood and empirical Bayesian estimation

Optimal approximating Markov chains for Bayesian inference

Accelerating delayed-acceptance Markov chain Monte Carlo algorithms

On the Computational Complexity of Metropolis-Adjusted Langevin Algorithms for Bayesian Posterior Sampling

Using Autodiff to Estimate Posterior Moments, Marginals and Samples