Abstract:Models with random effects, such as generalised linear mixed models (GLMMs), are often used for analysing clustered data. Parameter inference with these models is difficult because of the presence of cluster-specific random effects, which must be integrated out when evaluating the likelihood function. Here, we propose a sequential variational Bayes algorithm, called Recursive Variational Gaussian Approximation for Latent variable models (R-VGAL), for estimating parameters in GLMMs. The R-VGAL algorithm operates on the data sequentially, requires only a single pass through the data, and can provide parameter updates as new data are collected without the need of re-processing the previous data. At each update, the R-VGAL algorithm requires the gradient and Hessian of a "partial" log-likelihood function evaluated at the new observation, which are generally not available in closed form for GLMMs. To circumvent this issue, we propose using an importance-sampling-based approach for estimating the gradient and Hessian via Fisher's and Louis' identities. We find that R-VGAL can be unstable when traversing the first few data points, but that this issue can be mitigated by introducing a damping factor in the initial steps of the algorithm. Through illustrations on both simulated and real datasets, we show that R-VGAL provides good approximations to posterior distributions, that it can be made robust through damping, and that it is computationally efficient.

Vector Generalized Linear Models: A Gaussian Copula Approach

A Unified Bayesian Inference Framework for Generalized Linear Models

Statistical Inference for High-Dimensional Generalized Linear Models With Binary Outcomes

A Variational Approach for Modeling High-dimensional Spatial Generalized Linear Mixed Models

Decorrelated Empirical Likelihood for Generalized Linear Models with High-Dimensional Longitudinal Data

Marginally Interpretable Generalized Linear Mixed Models

A Bivariate Copula Additive Model for Location, Scale and Shape

Covariate-adjusted response-adaptive designs for generalized linear models

Covariance Model with General Linear Structure and Divergent Parameters

A convex formulation of covariate-adjusted Gaussian graphical models via natural parametrization

High-Dimensional Gaussian Graphical Regression Models with Covariates

R-VGAL: a sequential variational Bayes algorithm for generalised linear mixed models

On the properties of Gaussian Copula Mixture Models

Multivariate Generalised Linear Mixed Models With Graphical Latent Covariance Structure

Vector copulas

Inference for Gaussian Processes with Matérn Covariogram on Compact Riemannian Manifolds

Generalized linear models with spatial dependence and a functional covariate

Variable Sigma Gaussian Processes: an Expectation Propagation Perspective

Structured Variational Approximations with Skew Normal Decomposable Graphical Models and Implicit Copulas

Variable Selection for High-dimensional Generalized Linear Models using an Iterated Conditional Modes/Medians Algorithm

High-dimensional copula variational approximation through transformation