Abstract:We present a variational Bayesian (VB) implementation of block-sparse Bayesian learning (BSBL), which approximates the posterior probability density function (PDF) of the latent variables a factorzied proxy PDFs. The prior distribution of the BSBL hyperparameters is selected to be the generalized inverse Gaussian distribution. This choice unifies commonly used hyperpriors, e.g. the Gamma distribution, inverse Gamma distribution, and Jeffrey's improper prior. The resulting variational BSBL (VA-BSBL) algorithm updates in an iterative manner the proxi PDFs of the weights and hyperparameters. The proxy PDF of the weights only depends on the proxy PDFs of the hyperparameters through the means of the latter PDFs. We adopt a scheduling of the iterations in which the proxi PDF of a single hyperparameter and the proxi PDF of the weights are cyclically updated ad infinitum. The resulting sequence of means of the hyperparameter proxi PDFs computed in this way can be expressed as a nonlinear first-order recurrence relation. Hence, the fixed points of the recurrence relation can be used for single-step check for convergence of this sequence. Including this check procedure in the VA-BSBL, we obtain a fast version of the algorithm, coined fast-BSBL (F-BSBL), which shows a two-order-of-magnitude faster runtime. Additionally, we analyse a necessary condition for the existence of fixed points and show that only certain improper generalized inverse Gaussian hyperpriors induce a sparse estimate of the weights. Finally, we generalize this equivalence to BSBL. Specifically, we show that the presented VA-BSBL is equivalent to expectation-maximization (EM)-based Type-II-BSBL. As a result, the fast versions of them, i.e. FBSBL for the former and Type-II-BSBL using coordinate ascent for the latter, are equivalent as well. This result allows for reinterpreting previous works on BSBL within our framework.

Fast Bayesian inference in a class of sparse linear mixed effects models

Fast Laplace Approximation for Sparse Bayesian Spike and Slab Models

Variational Bayesian Inference in High-Dimensional Linear Mixed Models

Inference algorithms and learning theory for Bayesian sparse factor analysis

Dynamic sparsity on dynamic regression models

Robust Bayesian nonparametric variable selection for linear regression

Bayesian variable selection in linear regression models with instrumental variables

Fast post-process Bayesian inference with Variational Sparse Bayesian Quadrature

Posterior contraction rates in a sparse non-linear mixed-effects model

Bayesian high-dimensional covariate selection in non-linear mixed-effects models using the SAEM algorithm

Bayesian estimation for longitudinal data in a joint model with HPCs

Variational Inference for Sparse Poisson Regression

A fast asynchronous MCMC sampler for sparse Bayesian inference

Fast approximate inference for multivariate longitudinal data

Empirical Bayes inference in sparse high-dimensional generalized linear models

Bayesian High-dimensional Linear Regression with Sparse Projection-posterior

Spike-and-Slab LASSO Generalized Additive Models and Scalable Algorithms for High-Dimensional Data Analysis

Fast Variational Block-Sparse Bayesian Learning

The spike-and-slab lasso and scalable algorithm to accommodate multinomial outcomes in variable selection problems

Bayesian Variable Selection for Mixed Effects Model with Shrinkage Prior.

Sparse estimation of linear model via Bayesian method