Abstract:Gaussian approximations are routinely employed in Bayesian statistics to ease inference when the target posterior is intractable. Although these approximations are asymptotically justified by Bernstein-von Mises type results, in practice the expected Gaussian behavior may poorly represent the shape of the posterior, thus affecting approximation accuracy. Motivated by these considerations, we derive an improved class of closed-form approximations of posterior distributions which arise from a new treatment of a third-order version of the Laplace method yielding approximations in a tractable family of skew-symmetric distributions. Under general assumptions which account for misspecified models and non-i.i.d. settings, this family of approximations is shown to have a total variation distance from the target posterior whose rate of convergence improves by at least one order of magnitude the one established by the classical Bernstein-von Mises theorem. Specializing this result to the case of regular parametric models shows that the same improvement in approximation accuracy can be also derived for polynomially bounded posterior functionals. Unlike other higher-order approximations, our results prove that it is possible to derive closed-form and valid densities which are expected to provide, in practice, a more accurate, yet similarly-tractable, alternative to Gaussian approximations of the target posterior, while inheriting its limiting frequentist properties. We strengthen such arguments by developing a practical skew-modal approximation for both joint and marginal posteriors that achieves the same theoretical guarantees of its theoretical counterpart by replacing the unknown model parameters with the corresponding MAP estimate. Empirical studies confirm that our theoretical results closely match the remarkable performance observed in practice, even in finite, possibly small, sample regimes.

On the UMVUE and Closed-Form Bayes Estimator for $Pr(X<Y<Z)$ and its Generalizations

Empirical Bayes Estimators of Risk Premium under Variance Related Premium Principle

A Unified Bayesian Inference Framework for Generalized Linear Models

Bayesian estimation and prediction for certain mixtures

A Parzen-based distance between probability measures as an alternative of summary statistics in Approximate Bayesian Computation

Generalized Bhattacharyya and Chernoff upper bounds on Bayes error using quasi-arithmetic means

Generalized multi-view model: Adaptive density estimation under low-rank constraints

Empirical Bayes in Bayesian learning: understanding a common practice

The Bridged Posterior: Optimization, Profile Likelihood and a New Approach to Generalized Bayes

Quasi-maximum likelihood estimation and penalized estimation under non-standard conditions

Skewed Bernstein-von Mises theorem and skew-modal approximations

A Note on the Convergence Rates for Empirical Bayes Estimators of Parameters in Multiple-Parameter Exponential Families

Some Further Results on the Minimum Error Entropy Estimation.

An elementary approach for minimax estimation of Bernoulli proportion in the restricted parameter space

Bayesian Quantile Estimation and Regression with Martingale Posteriors

The R2D2 Prior for Generalized Linear Mixed Models

Posterior propriety in Bayesian extreme value analyses using reference priors

Superiority of Empirical Bayes Estimator of the Mean Vector in Multivariate Normal Distribution

Conditional generalized quantiles based on expected utility model and equivalent characterization of properties

Generalized Median of Means Principle for Bayesian Inference

Simultaneous Inference for Multiple Proportions: A Multivariate Beta-Binomial Model