Abstract:Although variable selection is one of the most popular areas of modern statistical research, much of its development has taken place in the classical paradigm compared to the Bayesian counterpart. Somewhat surprisingly, both the paradigms have focussed almost completely on linear models, in spite of the vast scope offered by the model liberation movement brought about by modern advancements in studying real, complex phenomena. In this article, we investigate general Bayesian variable selection in models driven by Gaussian processes, which allows us to treat linear, non-linear and nonparametric models, in conjunction with even dependent setups, in the same vein. We consider the Bayes factor route to variable selection, and develop a general asymptotic theory for the Gaussian process framework in the "large p, large n" settings even with p>>n, establishing almost sure exponential convergence of the Bayes factor under appropriately mild conditions. The fixed p setup is included as a special case. To illustrate, we apply our general result to variable selection in linear regression, Gaussian process model with squared exponential covariance function accommodating the covariates, and a first order autoregressive process with time-varying covariates. We also follow up our theoretical investigations with ample simulation experiments in the above regression contexts and variable selection in a real, riboflavin data consisting of 71 observations but 4088 covariates. For implementation of variable selection using Bayes factors, we develop a novel and effective general-purpose transdimensional, transformation based Markov chain Monte Carlo algorithm, which has played a crucial role in our simulated and real data applications.

BIVAS: A scalable Bayesian method for bi-level variable selection with applications

varbvs: Fast Variable Selection for Large-scale Regression

A High-dimensional M-estimator Framework for Bi-level Variable Selection

Bayesian variable selection in linear regression models with instrumental variables

Multivariate Bayesian variable selection with application to multi-trait genetic fine mapping

Variational Bayesian Inference in High-Dimensional Linear Mixed Models

Scalable Bayesian bi-level variable selection in generalized linear models

Bayesian Variable Selection in Linear Regression in One Pass for Large Data Sets

A Novel Variational Bayesian Method for Variable Selection in Logistic Regression Models.

Bi-level variable selection via adaptive sparse group Lasso

Fast Variational Inference for Bayesian Factor Analysis in Single and Multi-Study Settings

A novel variable selection approach that iteratively optimizes variable space using weighted binary matrix sampling.

Variable selection for partially linear models via Bayesian subset modeling with diffusing prior

Application of Bayesian variable selection in logistic regression model

Bayes Factor Asymptotics for Variable Selection in the Gaussian Process Framework

Bayesian latent factor regression for multivariate functional data with variable selection

Scalable Bayesian variable selection for structured high‐dimensional data

Forward Variable Selection for Sparse Ultra-High Dimensional Varying Coefficient Models

Variable Selection with Scalable Bootstrap in Generalized Linear Model for Massive Data

A Variational Approach for Modeling High-dimensional Spatial Generalized Linear Mixed Models

Variational Inference for Large Bayesian Vector Autoregressions