Abstract:In the context of the expected-posterior prior (EPP) approach to Bayesian variable selection in linear models, we combine ideas from power-prior and unit-information-prior methodologies to simultaneously (a) produce a minimally-informative prior and (b) diminish the effect of training samples. The result is that in practice our power-expected-posterior (PEP) methodology is sufficiently insensitive to the size $n^{*}$ of the training sample, due to PEP’s unit-information construction, that one may take $n^{*}$ equal to the full-data sample size $n$ and dispense with training samples altogether. This promotes stability of the resulting Bayes factors, removes the arbitrariness arising from individual training-sample selections, and greatly increases computational speed, allowing many more models to be compared within a fixed CPU budget. We find that, under an independence Jeffreys (reference) baseline prior, the asymptotics of PEP Bayes factors are equivalent to those of Schwartz’s Bayesian Information Criterion (BIC), ensuring consistency of the PEP approach to model selection. Our PEP prior, due to its unit-information structure, leads to a variable-selection procedure that — in our empirical studies — (1) is systematically more parsimonious than the basic EPP with minimal training sample, while sacrificing no desirable performance characteristics to achieve this parsimony; (2) is robust to the size of the training sample, thus enjoying the advantages described above arising from the avoidance of training samples altogether; and (3) identifies maximum-a-posteriori models that achieve better out-of-sample predictive performance than that provided by standard EPPs, the $g$-prior, the hyper-$g$ prior, non-local priors, the Least Absolute Shrinkage and Selection Operator (LASSO) and Smoothly-Clipped Absolute Deviation (SCAD) methods.

Model Prior Distribution for Variable Selection in Linear Regression Models

Faithful Variable Screening for High-Dimensional Convex Regression

Bayesian variable selection in linear regression models with instrumental variables

Variable Selection for Generalized Varying Coefficient Partially Linear Models with Diverging Number of Parameters

A Novel Variational Bayesian Method for Variable Selection in Logistic Regression Models.

Bayesian Regression Using a Prior on the Model Fit: The R2-D2 Shrinkage Prior

Variable selection for partially linear models via Bayesian subset modeling with diffusing prior

Bayesian Variable Selection with Related Predictors

The R2D2 Prior for Generalized Linear Mixed Models

A weakly informative default prior distribution for logistic and other regression models

A Mixture of g-priors for Variable Selection when the Number of Regressors Grows with the Sample Size

Conditional sufficient variable selection with prior information

Using reference models in variable selection

Power-Expected-Posterior Priors for Variable Selection in Gaussian Linear Models

Expressing and visualizing model uncertainty in Bayesian variable selection using Cartesian credible sets

Regularization and Variable Selection with Copula Prior

Bayesian variable selection in linear regression models with non-normal errors

Robust Bayesian nonparametric variable selection for linear regression

A Unified Variable Selection Approach for Varying Coefficient Models

Model uncertainty quantification in Cox regression

Variable Selection in High-Dimensional Sparse Multiresponse Linear Regression Models