Abstract:With high-dimensional data emerging in various domains, sparse logistic regression models have gained much interest of researchers. Variable selection plays a key role in both improving the prediction accuracy and enhancing the interpretability of built models. Bayesian variable selection approaches enjoy many advantages such as high selection accuracy, easily incorporating many kinds of prior knowledge and so on. Because Bayesian methods generally make inference from the posterior distribution with Markov Chain Monte Carlo (MCMC) techniques, however, they become intractable in high-dimensional situations due to the large searching space. To address this issue, a novel variational Bayesian method for variable selection in high-dimensional logistic regression models is presented. The proposed method is based on the indicator model in which each covariate is equipped with a binary latent variable indicating whether it is important. The Bernoulli-type prior is adopted for the latent indicator variable. As for the specification of the hyperparameter in the Bernoulli prior, we provide two schemes to determine its optimal value so that the novel model can achieve sparsity adaptively. To identify important variables and make predictions, one efficient variational Bayesian approach is employed to make inference from the posterior distribution. The experiments conducted with both synthetic and some publicly available data show that the new method outperforms or is very competitive with some other popular counterparts.

Variational Discriminant Analysis with Variable Selection

Bayesian variable selection in linear regression models with instrumental variables

A Novel Variational Bayesian Method for Variable Selection in Logistic Regression Models.

Variable Selection for Latent Dirichlet Allocation

A hybrid deterministic–deterministic approach for high-dimensional Bayesian variable selection with a default prior

An Efficient Variable Selection Method for Predictive Discriminant Analysis

Variable Selection in Discriminant Partial Least-Squares Analysis

Effect Size Estimation and Misclassification Rate Based Variable Selection in Linear Discriminant Analysis

Variable Selection for High Dimensional Gaussian Copula Regression Model: an Adaptive Hypothesis Testing Procedure.

Nonstationary Gaussian Process Discriminant Analysis With Variable Selection for High-Dimensional Functional Data

Variable selection in discriminant analysis based on Gram-Schmidt process

Variable selection and updating in model-based discriminant analysis for high dimensional data with food authenticity applications

Optimal Feature Selection in High-Dimensional Discriminant Analysis

Variable Selection with Stepwise and Best Subset Approaches.

Variable Selection for Fisher Linear Discriminant Analysis Using the Modified Sequential Backward Selection Algorithm for the Microarray Data

Variable Selection in High-Dimensional Error-in-Variables Models Via Controlling the False Discovery Proportion

Variational Bayesian Inference in High-Dimensional Linear Mixed Models

Bayesian Variable Selection with Related Predictors

Variable selection in model-based clustering and discriminant analysis with a regularization approach

Covariance-enhanced Discriminant Analysis.

On Bic'S Selection Consistency for Discriminant Analysis