Abstract:Gaussian approximations are routinely employed in Bayesian statistics to ease inference when the target posterior is intractable. Although these approximations are asymptotically justified by Bernstein-von Mises type results, in practice the expected Gaussian behavior may poorly represent the shape of the posterior, thus affecting approximation accuracy. Motivated by these considerations, we derive an improved class of closed-form approximations of posterior distributions which arise from a new treatment of a third-order version of the Laplace method yielding approximations in a tractable family of skew-symmetric distributions. Under general assumptions which account for misspecified models and non-i.i.d. settings, this family of approximations is shown to have a total variation distance from the target posterior whose rate of convergence improves by at least one order of magnitude the one established by the classical Bernstein-von Mises theorem. Specializing this result to the case of regular parametric models shows that the same improvement in approximation accuracy can be also derived for polynomially bounded posterior functionals. Unlike other higher-order approximations, our results prove that it is possible to derive closed-form and valid densities which are expected to provide, in practice, a more accurate, yet similarly-tractable, alternative to Gaussian approximations of the target posterior, while inheriting its limiting frequentist properties. We strengthen such arguments by developing a practical skew-modal approximation for both joint and marginal posteriors that achieves the same theoretical guarantees of its theoretical counterpart by replacing the unknown model parameters with the corresponding MAP estimate. Empirical studies confirm that our theoretical results closely match the remarkable performance observed in practice, even in finite, possibly small, sample regimes.

Inadequacy of Interval Estimates Corresponding to Variational Bayesian Approximations.

Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians

Statistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selection

Matching Bayesian and frequentist coverage probabilities when using an approximate data covariance matrix

On the Approximation Accuracy of Gaussian Variational Inference

Improving Variational Autoencoder Estimation from Incomplete Data with Mixture Variational Families

The Bayesian Infinitesimal Jackknife for Variance

The minimum covariance determinant estimator for interval-valued data

EIGENVALUE DISTRIBUTIONS OF VARIANCE COMPONENTS ESTIMATORS IN HIGH-DIMENSIONAL RANDOM EFFECTS MODELS

Convergence and asymptotic normality of variational Bayesian approximations for exponential family models with missing values

Approximation Properties of Variational Bayes for Vector Autoregressions

Distortion estimates for approximate Bayesian inference

Variational inference: uncertainty quantification in additive models

Bayesian inference for a covariance matrix

Bayesian mixtures of common factor analyzers: Model, variational inference, and applications

High-dimensional copula variational approximation through transformation

Skewed Bernstein-von Mises theorem and skew-modal approximations

Covariances, Robustness, and Variational Bayes

Variational Bayes algorithm and posterior consistency of Ising model parameter estimation

Variational Bayes Approximations for Clustering via Mixtures of Normal Inverse Gaussian Distributions

Bivariate Maximum Likelihood Method for Fixed Effects Panel Interval-Valued Data Models