Abstract:Gaussian approximations are routinely employed in Bayesian statistics to ease inference when the target posterior is intractable. Although these approximations are asymptotically justified by Bernstein-von Mises type results, in practice the expected Gaussian behavior may poorly represent the shape of the posterior, thus affecting approximation accuracy. Motivated by these considerations, we derive an improved class of closed-form approximations of posterior distributions which arise from a new treatment of a third-order version of the Laplace method yielding approximations in a tractable family of skew-symmetric distributions. Under general assumptions which account for misspecified models and non-i.i.d. settings, this family of approximations is shown to have a total variation distance from the target posterior whose rate of convergence improves by at least one order of magnitude the one established by the classical Bernstein-von Mises theorem. Specializing this result to the case of regular parametric models shows that the same improvement in approximation accuracy can be also derived for polynomially bounded posterior functionals. Unlike other higher-order approximations, our results prove that it is possible to derive closed-form and valid densities which are expected to provide, in practice, a more accurate, yet similarly-tractable, alternative to Gaussian approximations of the target posterior, while inheriting its limiting frequentist properties. We strengthen such arguments by developing a practical skew-modal approximation for both joint and marginal posteriors that achieves the same theoretical guarantees of its theoretical counterpart by replacing the unknown model parameters with the corresponding MAP estimate. Empirical studies confirm that our theoretical results closely match the remarkable performance observed in practice, even in finite, possibly small, sample regimes.

Method-of-Moments Inference for GLMs and Doubly Robust Functionals under Proportional Asymptotics

Stochastic Generalized Method of Moments.

Moment-Based Adjustments of Statistical Inference in High-Dimensional Generalized Linear Models

Sub-optimality of the Naive Mean Field approximation for proportional high-dimensional Linear Regression

Doubly Robust Semiparametric Inference Using Regularized Calibrated Estimation with High-dimensional Data

Locally Robust Semiparametric Estimation

On large-sample estimation and testing via quadratic inference functions for correlated data

Moderate-Dimensional Inferences on Quadratic Functionals in Ordinary Least Squares

On Gmm Inference: Partial Identification, Identification Strength, And Nonstandard Asymptotics

Inference for possibly misspecified generalized linear models with nonpolynomial-dimensional nuisance parameters

Precise Error Analysis of Regularized M-estimators in High-dimensions

Bayes-Optimal Estimation in Generalized Linear Models via Spatial Coupling

Marginalised Normal Regression: Unbiased curve fitting in the presence of x-errors

Advances in Bayesian model selection consistency for high-dimensional generalized linear models

Statistical Inference on High Dimensional Gaussian Graphical Regression Models

Simultaneous Nonparametric Regression Analysis of Sparse Longitudinal Data

Inference on Strongly Identified Functionals of Weakly Identified Functions

Skewed Bernstein-von Mises theorem and skew-modal approximations

Average partial effect estimation using double machine learning

Estimating Stochastic Linear Combination of Non-Linear Regressions Efficiently and Scalably