Abstract:Although variable selection is one of the most popular areas of modern statistical research, much of its development has taken place in the classical paradigm compared to the Bayesian counterpart. Somewhat surprisingly, both the paradigms have focussed almost completely on linear models, in spite of the vast scope offered by the model liberation movement brought about by modern advancements in studying real, complex phenomena. In this article, we investigate general Bayesian variable selection in models driven by Gaussian processes, which allows us to treat linear, non-linear and nonparametric models, in conjunction with even dependent setups, in the same vein. We consider the Bayes factor route to variable selection, and develop a general asymptotic theory for the Gaussian process framework in the "large p, large n" settings even with p>>n, establishing almost sure exponential convergence of the Bayes factor under appropriately mild conditions. The fixed p setup is included as a special case. To illustrate, we apply our general result to variable selection in linear regression, Gaussian process model with squared exponential covariance function accommodating the covariates, and a first order autoregressive process with time-varying covariates. We also follow up our theoretical investigations with ample simulation experiments in the above regression contexts and variable selection in a real, riboflavin data consisting of 71 observations but 4088 covariates. For implementation of variable selection using Bayes factors, we develop a novel and effective general-purpose transdimensional, transformation based Markov chain Monte Carlo algorithm, which has played a crucial role in our simulated and real data applications.

A Bayesian approach for variable selection in mixture of logistic regressions with Pólya-Gamma data augmentation

Bayesian inference for logistic models using Polya-Gamma latent variables

Bayesian estimation and prediction for certain mixtures

Bayesian variable selection in linear regression models with instrumental variables

Bayesian Variable Selection for Single Index Logistic Model

A double Pólya-Gamma data augmentation scheme for a hierarchical Negative Binomial - Binomial data model

Robust Bayesian nonparametric variable selection for linear regression

A Mixture of g-priors for Variable Selection when the Number of Regressors Grows with the Sample Size

Bayesian Variable Selection for Probit Mixed Models Applied to Gene Selection

Robust Bayesian model selection for heavy-tailed linear regression using finite mixtures

On the "Poisson Trick" and its Extensions for Fitting Multinomial Regression Models

Fully Bayesian logistic regression with hyper-LASSO priors for high-dimensional feature selection

Bayesian approaches to variable selection in mixture models with application to disease clustering

Improved Bayesian Logistic Supervised Topic Models with Data Augmentation.

Application of Bayesian variable selection in logistic regression model

Gaussian Mixture Regression model with logistic weights, a penalized maximum likelihood approach

Bayesian variable selection in linear regression models with non-normal errors

Bayesian Variable Selection in Linear Regression in One Pass for Large Data Sets

A novel Bayesian method for variable selection and estimation in binary quantile regression

Convergence analysis of data augmentation algorithms for Bayesian robust multivariate linear regression with incomplete data

Bayes Factor Asymptotics for Variable Selection in the Gaussian Process Framework