Abstract:Doubly-intractable posterior distributions arise in many applications of statistics concerned with discrete and dependent data, including physics, spatial statistics, machine learning, the social sciences, and other fields. A specific example is psychometrics, which has adapted high-dimensional Ising models from machine learning, with a view to studying the interactions among binary item responses in educational assessments. To estimate high-dimensional Ising models from educational assessment data, $\ell_1$-penalized nodewise logistic regressions have been used. Theoretical results in high-dimensional statistics show that $\ell_1$-penalized nodewise logistic regressions can recover the true interaction structure with high probability, provided that certain assumptions are satisfied. Those assumptions are hard to verify in practice and may be violated, and quantifying the uncertainty about the estimated interaction structure and parameter estimators is challenging. We propose a Bayesian approach that helps quantify the uncertainty about the interaction structure and parameters without requiring strong assumptions, and can be applied to Ising models with thousands of parameters. We demonstrate the advantages of the proposed Bayesian approach compared with $\ell_1$-penalized nodewise logistic regressions by simulation studies and applications to small and large educational data sets with up to 2,485 parameters. Among other things, the simulation studies suggest that the Bayesian approach is more robust against model misspecification due to omitted covariates than $\ell_1$-penalized nodewise logistic regressions.

Fully Bayesian logistic regression with hyper-LASSO priors for high-dimensional feature selection

High-dimensional Feature Selection Using Hierarchical Bayesian Logistic Regression with Heavy-tailed Priors

Fully Bayesian Classification with Heavy-tailed Priors for Selection in High-dimensional Features with Grouping Structure

Application of Bayesian variable selection in logistic regression model

Bayesian estimation for longitudinal data in a joint model with HPCs

Robust adaptive LASSO in high-dimensional logistic regression

Regularized Multinomial Regression Method for Hyperspectral Data Classification Via Pathwise Coordinate Optimization

Sparse Bayesian Approach for Feature Selection

Bayesian Model Selection in High-Dimensional Settings

High-dimensional classification by sparse logistic regression

The spike-and-slab lasso and scalable algorithm to accommodate multinomial outcomes in variable selection problems

On high-dimensional classification by sparse generalized Bayesian logistic regression

Penalized Bayesian forward continuation ratio model with application to high-dimensional data with discrete survival outcomes

Bayesian Model Selection for High-Dimensional Ising Models, With Applications to Educational Data

Bayesian High-dimensional Linear Regression with Sparse Projection-posterior

Variable selection for partially linear models via Bayesian subset modeling with diffusing prior

Bayesian Variable Selection for Single Index Logistic Model

Variable Selection Using Nonlocal Priors in High-Dimensional Generalized Linear Models With Application to fMRI Data Analysis

Simultaneous Dimension Reduction and Variable Selection for Multinomial Logistic Regression

Neuronized Priors for Bayesian Sparse Linear Regression

Sparse Bayesian Hierarchical Modeling of High-dimensional Clustering Problems