Abstract:An important feature of Bayesian statistics is the opportunity to do sequential inference: the posterior distribution obtained after seeing a dataset can be used as prior for a second inference. However, when Monte Carlo sampling methods are used for inference, we only have a set of samples from the posterior distribution. To do sequential inference, we then either have to evaluate the second posterior at only these locations and reweight the samples accordingly, or we can estimate a functional description of the posterior probability distribution from the samples and use that as prior for the second inference. Here, we investigated to what extent we can obtain an accurate joint posterior from two datasets if the inference is done sequentially rather than jointly, under the condition that each inference step is done using Monte Carlo sampling. To test this, we evaluated the accuracy of kernel density estimates, Gaussian mixtures, mixtures of factor analyzers, vine copulas and Gaussian processes in approximating posterior distributions, and then tested whether these approximations can be used in sequential inference. In low dimensionality, Gaussian processes are more accurate, whereas in higher dimensionality Gaussian mixtures, mixtures of factor analyzers or vine copulas perform better. In our test cases of sequential inference, using posterior approximations gives more accurate results than direct sample reweighting, but joint inference is still preferable over sequential inference whenever possible. Since the performance is case-specific, we provide an R package <em>mvdens</em> with a unified interface for the density approximation methods.

A fast algorithm for sampling from the posterior of a von Mises distribution

Novel sampling method for the von Mises–Fisher distribution

Posterior Sampling in High Dimension via Diffusion Processes

IID Sampling from Doubly Intractable Distributions

IID Sampling from Intractable Distributions

Using Autodiff to Estimate Posterior Moments, Marginals and Samples

Approximating multivariate posterior distribution functions from Monte Carlo samples for sequential Bayesian inference

Parallel Markov Chain Monte Carlo for Non-Gaussian Posterior Distributions

Sampling from manifold-restricted distributions using tangent bundle projections

Robust random walk-like Metropolis-Hastings algorithms for concentrating posteriors

Sampling from Dirichlet process mixture models with unknown concentration parameter: Mixing issues in large data implementations

Non-Log-Concave and Nonsmooth Sampling via Langevin Monte Carlo Algorithms

An efficient adaptive MCMC algorithm for Pseudo-Bayesian quantum tomography

Monte Carlo Approximation of Bayes Factors via Mixing with Surrogate Distributions

A variational approach to sampling in diffusion processes

Informed sub-sampling MCMC: approximate Bayesian inference for large datasets

Efficient computational strategies for doubly intractable problems with applications to Bayesian social networks

Graph-accelerated Markov Chain Monte Carlo using Approximate Samples

Sampling-Based Approaches to Calculating Marginal Densities

Geometry in sampling methods: A review on manifold MCMC and particle-based variational inference methods

An exact projection pursuit-based algorithm for multivariate two-sample nonparametric testing applicable to retrospective and group sequential studies