Abstract:Gaussian approximations are routinely employed in Bayesian statistics to ease inference when the target posterior is intractable. Although these approximations are asymptotically justified by Bernstein-von Mises type results, in practice the expected Gaussian behavior may poorly represent the shape of the posterior, thus affecting approximation accuracy. Motivated by these considerations, we derive an improved class of closed-form approximations of posterior distributions which arise from a new treatment of a third-order version of the Laplace method yielding approximations in a tractable family of skew-symmetric distributions. Under general assumptions which account for misspecified models and non-i.i.d. settings, this family of approximations is shown to have a total variation distance from the target posterior whose rate of convergence improves by at least one order of magnitude the one established by the classical Bernstein-von Mises theorem. Specializing this result to the case of regular parametric models shows that the same improvement in approximation accuracy can be also derived for polynomially bounded posterior functionals. Unlike other higher-order approximations, our results prove that it is possible to derive closed-form and valid densities which are expected to provide, in practice, a more accurate, yet similarly-tractable, alternative to Gaussian approximations of the target posterior, while inheriting its limiting frequentist properties. We strengthen such arguments by developing a practical skew-modal approximation for both joint and marginal posteriors that achieves the same theoretical guarantees of its theoretical counterpart by replacing the unknown model parameters with the corresponding MAP estimate. Empirical studies confirm that our theoretical results closely match the remarkable performance observed in practice, even in finite, possibly small, sample regimes.

Approximating the predictive distribution of the beta distribution with the local variational method

Variational Learning for Finite Beta-Liouville Mixture Models and Its Application

Approximation Properties of Variational Bayes for Vector Autoregressions

Predictive variational inference: Learn the predictively optimal posterior distribution

On the Approximation Accuracy of Gaussian Variational Inference

Variational Bayesian Approximation (VBA): Implementation and Comparison of Different Optimization Algorithms

Stick-Breaking Dependent Beta Processes with Variational Inference

Approximate Bayesian Computation via Classification

Skewed Bernstein-von Mises theorem and skew-modal approximations

A simple approximating formula of implied volatility in general local volatility model

A Deterministic Global Optimization Method for Variational Inference

Validated Variational Inference via Practical Posterior Error Bounds

Variational Linearized Laplace Approximation for Bayesian Deep Learning

A Probabilistic Approach to Nonparametric Local Volatility

One Sample Fits All: Approximating All Probabilistic Values Simultaneously and Efficiently

High-dimensional copula variational approximation through transformation

Variational Inference: A Review for Statisticians

On the properties of variational approximations of Gibbs posteriors

VARIATIONAL BAYES' METHOD FOR FUNCTIONS WITH APPLICATIONS TO SOME INVERSE PROBLEMS

VAE-KRnet and its applications to variational Bayes

$α$-Variational Inference with Statistical Guarantees