Abstract:Gaussian approximations are routinely employed in Bayesian statistics to ease inference when the target posterior is intractable. Although these approximations are asymptotically justified by Bernstein-von Mises type results, in practice the expected Gaussian behavior may poorly represent the shape of the posterior, thus affecting approximation accuracy. Motivated by these considerations, we derive an improved class of closed-form approximations of posterior distributions which arise from a new treatment of a third-order version of the Laplace method yielding approximations in a tractable family of skew-symmetric distributions. Under general assumptions which account for misspecified models and non-i.i.d. settings, this family of approximations is shown to have a total variation distance from the target posterior whose rate of convergence improves by at least one order of magnitude the one established by the classical Bernstein-von Mises theorem. Specializing this result to the case of regular parametric models shows that the same improvement in approximation accuracy can be also derived for polynomially bounded posterior functionals. Unlike other higher-order approximations, our results prove that it is possible to derive closed-form and valid densities which are expected to provide, in practice, a more accurate, yet similarly-tractable, alternative to Gaussian approximations of the target posterior, while inheriting its limiting frequentist properties. We strengthen such arguments by developing a practical skew-modal approximation for both joint and marginal posteriors that achieves the same theoretical guarantees of its theoretical counterpart by replacing the unknown model parameters with the corresponding MAP estimate. Empirical studies confirm that our theoretical results closely match the remarkable performance observed in practice, even in finite, possibly small, sample regimes.

Bernstein-von Mises Theorems for Functionals of Covariance Matrix

High-Dimensional Bernstein Von-Mises Theorems for Covariance and Precision Matrices

Bernstein - von Mises theorems for statistical inverse problems I: Schrödinger equation

Bernstein von-Mises Theorem for g-prior and nonlocal prior

Functional CLT for sample covariance matrices

Valid Credible Ellipsoids for Linear Functionals by a Renormalized Bernstein-von Mises Theorem

A Matrix Version of Chernoff Inequality

Covariance Matrix Inequalities for Functions of Beta Random Variables

Skewed Bernstein-von Mises theorem and skew-modal approximations

Bernstein approximation and beyond: proofs by means of elementary probability theory

Gram matrix of Bernstein basis: Properties and applications

A Matrix Bernstein Inequality for Strong Rayleigh Distributions

Bernstein-von Mises theorems for time evolution equations

On the Bernstein-Von Mises Theorem for High Dimensional Nonlinear Bayesian Inverse Problems

Misspecified Bernstein-Von Mises theorem for hierarchical models

Bivariate Covariance Functions of Pólya Type

A Bernstein-von Mises Theorem for Generalized Fiducial Distributions

Bernstein estimator for conditional copulas

Asymptotic analysis for covariance parameter estimation of Gaussian processes with functional inputs

Variants of Bernstein's theorem for variational integrals with linear and nearly linear growth

Bernstein -- von Mises theorems for statistical inverse problems II: Compound Poisson processes