Abstract:Gaussian approximations are routinely employed in Bayesian statistics to ease inference when the target posterior is intractable. Although these approximations are asymptotically justified by Bernstein-von Mises type results, in practice the expected Gaussian behavior may poorly represent the shape of the posterior, thus affecting approximation accuracy. Motivated by these considerations, we derive an improved class of closed-form approximations of posterior distributions which arise from a new treatment of a third-order version of the Laplace method yielding approximations in a tractable family of skew-symmetric distributions. Under general assumptions which account for misspecified models and non-i.i.d. settings, this family of approximations is shown to have a total variation distance from the target posterior whose rate of convergence improves by at least one order of magnitude the one established by the classical Bernstein-von Mises theorem. Specializing this result to the case of regular parametric models shows that the same improvement in approximation accuracy can be also derived for polynomially bounded posterior functionals. Unlike other higher-order approximations, our results prove that it is possible to derive closed-form and valid densities which are expected to provide, in practice, a more accurate, yet similarly-tractable, alternative to Gaussian approximations of the target posterior, while inheriting its limiting frequentist properties. We strengthen such arguments by developing a practical skew-modal approximation for both joint and marginal posteriors that achieves the same theoretical guarantees of its theoretical counterpart by replacing the unknown model parameters with the corresponding MAP estimate. Empirical studies confirm that our theoretical results closely match the remarkable performance observed in practice, even in finite, possibly small, sample regimes.

Kullback-leibler Approximation of Spectral Density Functions

Optimal Reduction of Multivariate Dirac Mixture Densities

Quantitative Error Analyses of Spectral Density Estimators Using Covariance Lags

A Convex Optimization Approach to ARMA Modeling

Approximate Optimal Estimation Based on Kullback–Leibler Divergence for Lossy Networks Without Acknowledgement

Filtering Problem for Functionals of Stationary Sequences

Minimal Kullback–Leibler Divergence for Constrained Lévy–Itô Processes

Distances between power spectral densities

Almost-sure quasi-optimal approximation in reproducing kernel Hilbert spaces

Testing Equality of Spectral Density Operators for Functional Processes

Optimal low-rank approximations of posteriors for linear Gaussian inverse problems on Hilbert spaces

Skewed Bernstein-von Mises theorem and skew-modal approximations

On the Approximation of Kernel functions

Spectral convergence of probability densities for forward problems in uncertainty quantification

An Iterative Nonlinear Filter Based on Posterior Distribution Approximation Via Penalized Kullback-Leibler Divergence Minimization

Instance-Optimal Private Density Estimation in the Wasserstein Distance

Approximation in Hilbert spaces of the Gaussian and related analytic kernels

Optimal Quantization for Distribution Synthesis

Improved Spectral Density Estimation via Explicit and Implicit Deflation

Linear cost and exponentially convergent approximation of Gaussian Matérn processes

Optimal State Estimation in the Presence of Non-Gaussian Uncertainty via Wasserstein Distance Minimization