Abstract:Bayesian models are generally computed with Markov Chain Monte Carlo (MCMC) methods. The main disadvantage about MCMC methods is the large number of iterations they need to sample the posterior distributions of model parameters, especially for large data sets. On the other hand, variable selection remains a challenging problem due to its combinatorial search space, where Bayesian models are a promising solution. In this work, we study how to accelerate Bayesian model computation for variable selection in linear regression. We propose a fast Gibbs sampler algorithm, a widely used MCMC method, that incorporates several optimizations. We use non-informative and conjugate prior distributions on several model parameters, which enable data set summarization in one pass exploiting an augmented set of sufficient statistics. Thereafter the algorithm can iterate in main memory. Sufficient statistics are indexed with a sparse binary vector to efficiently compute matrix projections based on selected variables. Discovered variable subsets probabilities, selecting and discarding each variable, are stored on a hash table for fast retrieval in future iterations. We study how to integrate our algorithm into a database management system (DBMS), exploiting aggregate User-Defined Functions for parallel data summarization and stored procedures to manipulate matrices with arrays. An experimental evaluation with real data sets evaluates accuracy and time performance, comparing our DBMS-based algorithm, with the R package. Our algorithm is shown to produce accurate results, scale linearly on data set size and run orders of magnitude faster than the R package.

Sparse Bayesian Variable Selection in Kernel Probit Model for Analyzing High-Dimensional Data

Process Monitoring with Sparse Bayesian Model for Industrial Methanol Distillation

Bayesian High-dimensional Linear Regression with Sparse Projection-posterior

High-dimensional Grouped-regression using Bayesian Sparse Projection-posterior

Bayesian variable selection in linear regression models with instrumental variables

Bayesian estimation for longitudinal data in a joint model with HPCs

Bayesian Analysis for Over-parameterized Linear Model without Sparsity

Sparse estimation of linear model via Bayesian method

Variable selection using axis-aligned random projections for partial least-squares regression

Sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm

Empirical Bayes inference in sparse high-dimensional generalized linear models

Two-Step Mixed-Type Multivariate Bayesian Sparse Variable Selection with Shrinkage Priors

Bayesian Grouping-Gibbs Sampling Estimation of High-dimensional Linear Model with Non-sparsity

On high-dimensional classification by sparse generalized Bayesian logistic regression

Application of Bayesian variable selection in logistic regression model

Fully Bayesian logistic regression with hyper-LASSO priors for high-dimensional feature selection

Sparse Bayesian Approach for Feature Selection

Bayesian Model Selection in High-Dimensional Settings

Sparse Bayesian Hierarchical Modeling of High-dimensional Clustering Problems

Robust Bayesian nonparametric variable selection for linear regression

Bayesian Variable Selection in Linear Regression in One Pass for Large Data Sets