Abstract:Abstract Markov chain Monte Carlo (MCMC)-based simulation approaches are by far the most common method in Bayesian inference to access the posterior distribution. Recently, motivated by successes in machine learning, variational inference (VI) has gained in interest in statistics since it promises a computationally efficient alternative to MCMC enabling approximate access to the posterior. Classical approaches such as mean-field VI (MFVI), however, are based on the strong mean-field assumption for the approximate posterior where parameters or parameter blocks are assumed to be mutually independent. As a consequence, parameter uncertainties are often underestimated and alternatives such as semi-implicit VI (SIVI) have been suggested to avoid the mean-field assumption and to improve uncertainty estimates. SIVI uses a hierarchical construction of the variational parameters to restore parameter dependencies and relies on a highly flexible implicit mixing distribution whose probability density function is not analytic but samples can be taken via a stochastic procedure. With this paper, we investigate how different forms of VI perform in semiparametric additive regression models as one of the most important fields of application of Bayesian inference in statistics. A particular focus is on the ability of the rivalling approaches to quantify uncertainty, especially with correlated covariates that are likely to aggravate the difficulties of simplifying VI assumptions. Moreover, we propose a method, where we combine both advantages of MFVI and SIVI and compare its performance. The different VI approaches are studied in comparison with MCMC in simulations and an application to tree height models of douglas fir based on a large-scale forestry data set.

Fast robustness quantification with variational Bayes

Covariances, Robustness, and Variational Bayes

Variational Approximation of Factor Stochastic Volatility Models

Efficient Variational Bayesian Model Updating by Bayesian Active Learning

Fitting Structural Equation Models via Variational Approximations

Variational inference for large-scale models of discrete choice

Variational inference: uncertainty quantification in additive models

Parameter Expanded Variational Bayesian Methods

Variational Inference for Large Bayesian Vector Autoregressions

Seemingly unrelated regression with measurement error: estimation via Markov Chain Monte Carlo and mean field variational Bayes approximation

Efficient Selection Between Hierarchical Cognitive Models: Cross-validation With Variational Bayes

Variational Bayesian Inference for Quantile Regression Models with Nonignorable Missing Data

Variational Bayes for high-dimensional proportional hazards models with applications within gene expression

Uncertainty Quantification for Markov Processes via Variational Principles and Functional Inequalities

Validated Variational Inference via Practical Posterior Error Bounds

Variational Bayesian inference with stochastic search

Variational Bayesian hierarchical regression for data analysis

Bayesian Methods for Empirical Macroeconomics with Big Data

The Bayesian Infinitesimal Jackknife for Variance