Abstract:Bayesian models are generally computed with Markov Chain Monte Carlo (MCMC) methods. The main disadvantage about MCMC methods is the large number of iterations they need to sample the posterior distributions of model parameters, especially for large data sets. On the other hand, variable selection remains a challenging problem due to its combinatorial search space, where Bayesian models are a promising solution. In this work, we study how to accelerate Bayesian model computation for variable selection in linear regression. We propose a fast Gibbs sampler algorithm, a widely used MCMC method, that incorporates several optimizations. We use non-informative and conjugate prior distributions on several model parameters, which enable data set summarization in one pass exploiting an augmented set of sufficient statistics. Thereafter the algorithm can iterate in main memory. Sufficient statistics are indexed with a sparse binary vector to efficiently compute matrix projections based on selected variables. Discovered variable subsets probabilities, selecting and discarding each variable, are stored on a hash table for fast retrieval in future iterations. We study how to integrate our algorithm into a database management system (DBMS), exploiting aggregate User-Defined Functions for parallel data summarization and stored procedures to manipulate matrices with arrays. An experimental evaluation with real data sets evaluates accuracy and time performance, comparing our DBMS-based algorithm, with the R package. Our algorithm is shown to produce accurate results, scale linearly on data set size and run orders of magnitude faster than the R package.

Variable selection for regression splines using hierarchical sparseness prior

Bayesian variable selection in linear regression models with instrumental variables

Bayesian Variable Selection with Related Predictors

Sparse Variable Selection on High Dimensional Heterogeneous Data With Tree Structured Responses

Hierarchical sparsity priors for regression models

Disjunct Support Spike and Slab Priors for Variable Selection in Regression under Quasi-sparseness

Variable Selection for Additive Model Via Cumulative Ratios of Empirical Strengths Total

Bayesian Sparse Global-Local Shrinkage Regression for Selection of Grouped Variables

Spline estimator for ultra-high dimensional partially linear varying coefficient models

Variable selection for partially linear models via Bayesian subset modeling with diffusing prior

A Transparent and Nonlinear Method for Variable Selection

Forward Variable Selection for Sparse Ultra-High Dimensional Varying Coefficient Models

Variable Selection for the Partial Linear Single-Index Model

Group Regularized Estimation under Structural Hierarchy

A High-dimensional M-estimator Framework for Bi-level Variable Selection

Forward variable selection for sparse ultra-high-dimensional generalized varying coefficient models

Spline regression with automatic knot selection

Synergizing Roughness Penalization and Basis Selection in Bayesian Spline Regression

Variable selection for semiparametric varying-coefficient spatial autoregressive models with a diverging number of parameters

Dynamic sparsity on dynamic regression models

Bayesian Variable Selection in Linear Regression in One Pass for Large Data Sets