Abstract:Sparse linear (or generalized linear) models combine a standard likelihood function with a sparse prior on the unknown coefficients. These priors can conveniently be expressed as a maximization over zero-mean Gaussians with different variance hyperparameters. Standard MAP estimation (Type I) involves maximizing over both the hyperparameters and coefficients, while an empirical Bayesian alternative (Type II) first marginalizes the coefficients and then maximizes over the hyperparameters, leading to a tractable posterior approximation. The underlying cost functions can be related via a dual-space framework from Wipf et al. (2011), which allows both the Type I or Type II objectives to be expressed in either coefficient or hyperparmeter space. This perspective is useful because some analyses or extensions are more conducive to development in one space or the other. Herein we consider the estimation of a trade-off parameter balancing sparsity and data fit. As this parameter is effectively a variance, natural estimators exist by assessing the problem in hyperparameter (variance) space, transitioning natural ideas from Type II to solve what is much less intuitive for Type I. In contrast, for analyses of update rules and sparsity properties of local and global solutions, as well as extensions to more general likelihood models, we can leverage coefficient-space techniques developed for Type I and apply them to Type II. For example, this allows us to prove that Type II-inspired techniques can be successful recovering sparse coefficients when unfavorable restricted isometry properties (RIP) lead to failure of popular L1 reconstructions. It also facilitates the analysis of Type II when non-Gaussian likelihood models lead to intractable integrations.

Sparse estimation of linear model via Bayesian method

Bayesian High-dimensional Linear Regression with Sparse Projection-posterior

Bayesian Sparse Global-Local Shrinkage Regression for Selection of Grouped Variables

On The Sparse Bayesian Learning Of Linear Models

Process Monitoring with Sparse Bayesian Model for Industrial Methanol Distillation

Neuronized Priors for Bayesian Sparse Linear Regression

Bayesian Variable Selection and Estimation Based on Global-Local Shrinkage Priors

Tail-adaptive Bayesian shrinkage

Bayesian Analysis for Over-parameterized Linear Model without Sparsity

Bayesian estimation for longitudinal data in a joint model with HPCs

On Sparse Estimation for Semiparametric Linear Transformation Models

High-dimensional Grouped-regression using Bayesian Sparse Projection-posterior

Concentration of a Sparse Bayesian Model With Horseshoe Prior in Estimating High‐Dimensional Precision Matrix

Empirical Bayes inference in sparse high-dimensional generalized linear models

Bayesian Estimation Approach for Linear Regression Models with Linear Inequality Restrictions

Bayesian Grouping-Gibbs Sampling Estimation of High-dimensional Linear Model with Non-sparsity

Dynamic sparsity on dynamic regression models

Concentration of a sparse Bayesian model with Horseshoe prior in estimating high-dimensional precision matrix

Bayesian Regression Using a Prior on the Model Fit: The R2-D2 Shrinkage Prior

Dual-Space Analysis of the Sparse Linear Model