Abstract:Abstract Markov chain Monte Carlo (MCMC)-based simulation approaches are by far the most common method in Bayesian inference to access the posterior distribution. Recently, motivated by successes in machine learning, variational inference (VI) has gained in interest in statistics since it promises a computationally efficient alternative to MCMC enabling approximate access to the posterior. Classical approaches such as mean-field VI (MFVI), however, are based on the strong mean-field assumption for the approximate posterior where parameters or parameter blocks are assumed to be mutually independent. As a consequence, parameter uncertainties are often underestimated and alternatives such as semi-implicit VI (SIVI) have been suggested to avoid the mean-field assumption and to improve uncertainty estimates. SIVI uses a hierarchical construction of the variational parameters to restore parameter dependencies and relies on a highly flexible implicit mixing distribution whose probability density function is not analytic but samples can be taken via a stochastic procedure. With this paper, we investigate how different forms of VI perform in semiparametric additive regression models as one of the most important fields of application of Bayesian inference in statistics. A particular focus is on the ability of the rivalling approaches to quantify uncertainty, especially with correlated covariates that are likely to aggravate the difficulties of simplifying VI assumptions. Moreover, we propose a method, where we combine both advantages of MFVI and SIVI and compare its performance. The different VI approaches are studied in comparison with MCMC in simulations and an application to tree height models of douglas fir based on a large-scale forestry data set.

Fitting Structural Equation Models via Variational Approximations

Variational Bayes for Mixture of Gaussian Structural Equation Models

Variational inference for large-scale models of discrete choice

Efficient Bayesian Structural Equation Modeling in Stan

Ultra-Fast Approximate Inference Using Variational Functional Mixed Models

Efficient Variational Bayesian Model Updating by Bayesian Active Learning

Variational Approximation of Factor Stochastic Volatility Models

A Variational Approach to Bayesian Phylogenetic Inference

Fast robustness quantification with variational Bayes

Bayesian Model Selection via Mean-Field Variational Approximation

Bayesian Structural Model Updating with Multimodal Variational Autoencoder

Bayesian Inference Via Variational Approximation for Collaborative Filtering.

High-dimensional copula variational approximation through transformation

Variational Bayesian hierarchical regression for data analysis

Variational inference: uncertainty quantification in additive models

Multilevel Latent Differential Structural Equation Model with Short Time Series and Time-Varying Covariates: A Comparison of Frequentist and Bayesian Estimators

Seemingly unrelated regression with measurement error: estimation via Markov Chain Monte Carlo and mean field variational Bayes approximation

Variational Inference for Large Bayesian Vector Autoregressions

Variational Gibbs Inference for Statistical Model Estimation from Incomplete Data

Variational Bayesian Variable Selection for High-Dimensional Hidden Markov Models

Structured Optimal Variational Inference for Dynamic Latent Space Models