Abstract:An important feature of Bayesian statistics is the opportunity to do sequential inference: the posterior distribution obtained after seeing a dataset can be used as prior for a second inference. However, when Monte Carlo sampling methods are used for inference, we only have a set of samples from the posterior distribution. To do sequential inference, we then either have to evaluate the second posterior at only these locations and reweight the samples accordingly, or we can estimate a functional description of the posterior probability distribution from the samples and use that as prior for the second inference. Here, we investigated to what extent we can obtain an accurate joint posterior from two datasets if the inference is done sequentially rather than jointly, under the condition that each inference step is done using Monte Carlo sampling. To test this, we evaluated the accuracy of kernel density estimates, Gaussian mixtures, mixtures of factor analyzers, vine copulas and Gaussian processes in approximating posterior distributions, and then tested whether these approximations can be used in sequential inference. In low dimensionality, Gaussian processes are more accurate, whereas in higher dimensionality Gaussian mixtures, mixtures of factor analyzers or vine copulas perform better. In our test cases of sequential inference, using posterior approximations gives more accurate results than direct sample reweighting, but joint inference is still preferable over sequential inference whenever possible. Since the performance is case-specific, we provide an R package <em>mvdens</em> with a unified interface for the density approximation methods.

Stochastic Convergence Rates and Applications of Adaptive Quadrature in Bayesian Inference

Adaptive sampling-based quadrature rules for efficient Bayesian prediction

Efficient quadratures for high-dimensional Bayesian data assimilation

Central Limit Theorems of a Recursive Stochastic Algorithm with Applications to Adaptive Designs

Conditional Bayesian Quadrature

Exact gradient evaluation for adaptive quadrature approximate marginal likelihood in mixed models for grouped data

An automatic adaptive method to combine summary statistics in approximate Bayesian computation

A note on the accuracy of adaptive Gauss–Hermite quadrature

Small-sample Size Problems Solving Based on Incremental Learning: an Adaptive Bayesian Quadrature Approach

Achieving High Convergence Rates by Quasi-Monte Carlo and Importance Sampling for Unbounded Integrands

Parallel adaptive Bayesian quadrature for rare event estimation

Quasi-Monte Carlo and importance sampling methods for Bayesian inverse problems

Asymptotics of approximate Bayesian computation when summary statistics converge at heterogeneous rates

Approximating multivariate posterior distribution functions from Monte Carlo samples for sequential Bayesian inference

On Average-Case Error Bounds for Kernel-Based Bayesian Quadrature

Ideal Bayesian Spatial Adaptation

Randomized matrix-free quadrature: unified and uniform bounds for stochastic Lanczos quadrature and the kernel polynomial method

Convergence analysis of data augmentation algorithms for Bayesian robust multivariate linear regression with incomplete data

On the convergence of adaptive approximations for stochastic differential equations

Stochastic Approximation with Decision-Dependent Distributions: Asymptotic Normality and Optimality

Refined parallel adaptive Bayesian quadrature for estimating small failure probabilities