Abstract:Summary Structured additive regression models are perhaps the most commonly used class of models in statistical applications. It includes, among others, (generalized) linear models, (generalized) additive models, smoothing spline models, state space models, semiparametric regression, spatial and spatiotemporal models, log-Gaussian Cox processes and geostatistical and geoadditive models. We consider approximate Bayesian inference in a popular subset of structured additive regression models, latent Gaussian models, where the latent field is Gaussian, controlled by a few hyperparameters and with non-Gaussian response variables. The posterior marginals are not available in closed form owing to the non-Gaussian response variables. For such models, Markov chain Monte Carlo methods can be implemented, but they are not without problems, in terms of both convergence and computational time. In some practical applications, the extent of these problems is such that Markov chain Monte Carlo sampling is simply not an appropriate tool for routine analysis. We show that, by using an integrated nested Laplace approximation and its simplified version, we can directly compute very accurate approximations to the posterior marginals. The main benefit of these approximations is computational: where Markov chain Monte Carlo algorithms need hours or days to run, our approximations provide more precise estimates in seconds or minutes. Another advantage with our approach is its generality, which makes it possible to perform Bayesian analysis in an automatic, streamlined way, and to compute model comparison criteria and various predictive measures so that models can be compared and the model under study can be challenged.

Approximation Properties of Gaussian-binary Restricted Boltzmann Machines and Gaussian-binary Deep Belief Networks

Learning Gaussian Networks

Training Restricted Boltzmann Machines with Binary Synapses Using the Bayesian Learning Rule

Refinements of Approximation Results of Conditional Restricted Boltzmann Machines

Approximation by random networks with bounded number of layers

A Gaussian RBM with Binary Auxiliary Units

Probabilistic Inference of Bayesian Neural Networks with Generalized Expectation Propagation

Gaussian Approximation Theorems for Urn Models and Their Applications

Function Approximation by Random Neural Networks with a Bounded Number of Layers

Likelihood approximations via Gaussian approximate inference

Approximation Bounds for Norm Constrained Neural Networks with Applications to Regression and GANs

Normal approximation of Random Gaussian Neural Networks

Deep Narrow Boltzmann Machines are Universal Approximators

Classification of Data Generated by Gaussian Mixture Models Using Deep ReLU Networks

Gaussian Cardinality Restricted Boltzmann Machines.

Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations

A Non-negative VAE:the Generalized Gamma Belief Network

Approximated Anomalous Diffusion: Gaussian Mixture Score-based Generative Models

Bayesian leave-one-out cross-validation approximations for Gaussian latent variable models

On the capacity of deep generative networks for approximating distributions

Generalized Laplace Approximation