Abstract:Finding statistically significant high-order interaction features in predictive modeling is important but challenging task. The difficulty lies in the fact that, for a recent applications with high-dimensional covariates, the number of possible high-order interaction features would be extremely large. Identifying statistically significant features from such a huge pool of candidates would be highly challenging both in computational and statistical senses. To work with this problem, we consider a two stage algorithm where we first select a set of high-order interaction features by marginal screening, and then make statistical inferences on the regression model fitted only with the selected features. Such statistical inferences are called post-selection inference (PSI), and receiving an increasing attention in the literature. One of the seminal recent advancements in PSI literature is the works by Lee et al. where the authors presented an algorithmic framework for computing exact sampling distributions in PSI. A main challenge when applying their approach to our high-order interaction models is to cope with the fact that PSI in general depends not only on the selected features but also on the unselected features, making it hard to apply to our extremely high-dimensional high-order interaction models. The goal of this paper is to overcome this difficulty by introducing a novel efficient method for PSI. Our key idea is to exploit the underlying tree structure among high-order interaction features, and to develop a pruning method of the tree which enables us to quickly identify a group of unselected features that are guaranteed to have no influence on PSI. The experimental results indicate that the proposed method allows us to reliably identify statistically significant high-order interaction features with reasonable computational cost.

Partial profile score feature selection in high-dimensional generalized linear interaction models

A Sequential Feature Selection Procedure for High-Dimensional Cox Proportional Hazards Model

A semi-parametric approach to feature selection in high-dimensional linear regression models

Feature Selection with Conditional Mutual Information Considering Feature Interaction

Sequential Interaction Group Selection by the Principle of Correlation Search for High-Dimensional Interaction Models

Sequential profile Lasso for ultra-high-dimensional partially linear models

Feature screening and variable selection for partially linear models with ultrahigh-dimensional longitudinal data

Sequential Feature Screening for Generalized Linear Models with Sparse Ultra-High Dimensional Data

Sequential Lasso for feature selection with ultra-high dimensional feature space

Factor Profiling for Ultra High Dimensional Variable Selection

Rank-Based Sequential Feature Selection for High-Dimensional Accelerated Failure Time Models with Main and Interaction Effects

Profiled Forward Regression for Ultrahigh Dimensional Variable Screening in Semiparametric Partially Linear Models

Feature Screening for High-Dimensional Variable Selection in Generalized Linear Models

Feature Selection by Canonical Correlation Search in High-Dimensional Multiresponse Models with Complex Group Structures

Variable selection for partially linear models via partial correlation

Independent Feature Screening for Ultrahigh-Dimensional Models with Interactions

Interaction Pursuit with Feature Screening and Selection

Efficient Estimation for Marginal Generalized Partially Linear Single-Index Models with Longitudinal Data

Sequential Lasso Cum EBIC for Feature Selection with Ultra-High Dimensional Feature Space

Profile forward regression screening for ultra-high dimensional semiparametric varying coefficient partially linear models

An Efficient Post-Selection Inference on High-Order Interaction Models