Abstract:We present a variational Bayesian (VB) implementation of block-sparse Bayesian learning (BSBL), which approximates the posterior probability density function (PDF) of the latent variables a factorzied proxy PDFs. The prior distribution of the BSBL hyperparameters is selected to be the generalized inverse Gaussian distribution. This choice unifies commonly used hyperpriors, e.g. the Gamma distribution, inverse Gamma distribution, and Jeffrey's improper prior. The resulting variational BSBL (VA-BSBL) algorithm updates in an iterative manner the proxi PDFs of the weights and hyperparameters. The proxy PDF of the weights only depends on the proxy PDFs of the hyperparameters through the means of the latter PDFs. We adopt a scheduling of the iterations in which the proxi PDF of a single hyperparameter and the proxi PDF of the weights are cyclically updated ad infinitum. The resulting sequence of means of the hyperparameter proxi PDFs computed in this way can be expressed as a nonlinear first-order recurrence relation. Hence, the fixed points of the recurrence relation can be used for single-step check for convergence of this sequence. Including this check procedure in the VA-BSBL, we obtain a fast version of the algorithm, coined fast-BSBL (F-BSBL), which shows a two-order-of-magnitude faster runtime. Additionally, we analyse a necessary condition for the existence of fixed points and show that only certain improper generalized inverse Gaussian hyperpriors induce a sparse estimate of the weights. Finally, we generalize this equivalence to BSBL. Specifically, we show that the presented VA-BSBL is equivalent to expectation-maximization (EM)-based Type-II-BSBL. As a result, the fast versions of them, i.e. FBSBL for the former and Type-II-BSBL using coordinate ascent for the latter, are equivalent as well. This result allows for reinterpreting previous works on BSBL within our framework.

What problem does this paper attempt to address?

The paper aims to address several key issues in Block-Sparse Bayesian Learning (BSBL): 1. **Improving Algorithm Convergence Speed**: Traditional BSBL algorithms have a slow convergence speed when estimating hyperparameters, which limits their efficiency in practical applications. The paper proposes a Fast Variational BSBL algorithm (Fast-BSBL, F-BSBL), which significantly improves the convergence speed by introducing fast update rules. 2. **Unifying Different Hyperprior Distributions**: The paper uses the Generalized Inverse Gaussian Distribution (GIG) as the prior distribution for hyperparameters. This not only unifies commonly used hyperprior distributions (such as Gamma distribution, Inverse Gamma distribution, and Jeffrey's non-informative prior) but also allows for the analysis of the impact of these priors on weight estimation. 3. **Analyzing Conditions for Fixed Points**: The paper analyzes the conditions for the existence of fixed points and proves that under certain specific GIG hyperpriors, sparse weight estimates can be induced. This is crucial for achieving effective sparse signal reconstruction. 4. **Equivalence Analysis**: The paper demonstrates the equivalence between the Expectation-Maximization (EM) based Type-II-BSBL and the Variational BSBL (VA-BSBL). It further proves the equivalence between the fast version of F-BSBL and the coordinate ascent-based fast Type-II-BSBL. This result allows previous work on BSBL to be reinterpreted as variational message passing algorithms. Through these contributions, the paper not only enhances the efficiency and performance of BSBL algorithms but also provides a solid theoretical foundation for understanding and applying BSBL.

Fast Variational Block-Sparse Bayesian Learning

Fast Inverse-Free Generalized Sparse Bayesian Iearning Algorithm

An AMP-Based Low Complexity Generalized Sparse Bayesian Learning Algorithm

Unitary Approximate Message Passing for Sparse Bayesian Learning

Sparse Bayesian Learning Approach for Discrete Signal Reconstruction

Fast Inverse-Free Sparse Bayesian Learning Via Relaxed Evidence Lower Bound Maximization.

Fast post-process Bayesian inference with Variational Sparse Bayesian Quadrature

Hyperparameter Estimation for Sparse Bayesian Learning Models

Inference algorithms and learning theory for Bayesian sparse factor analysis

Towards Inversion-Free Sparse Bayesian Learning: A Universal Approach

Gohberg-Semencul Factorization-Based Fast Implementation of Sparse Bayesian Learning With a Fourier Dictionary

Sparse Bayesian Learning with Automatic-Weighting Laplace Priors for Sparse Signal Recovery.

Fast variational Bayesian learning for channel estimation with prior statistical information

Variance State Propagation for Structured Sparse Bayesian Learning

Fast Bayesian inference in a class of sparse linear mixed effects models

Covariance-Free Sparse Bayesian Learning

Fast Laplace Approximation for Sparse Bayesian Spike and Slab Models

Block Sparse Bayesian Learning: A Diversified Scheme

Bayesian Sparse Estimation Using Double Lomax Priors

Fast and robust Block-Sparse Bayesian learning for EEG source imaging