Abstract:High-order covariates interaction is considered into Lasso-type variable selection.We evaluate the significance of feature by considering their neighborhood dependency.Having too few features in not necessarily a good feature selection result.Some interactive features may be lost in the process of removing redundancy. Lasso-type feature selection has been demonstrated to be effective in handling high dimensional data. Most existing Lasso-type models over emphasize the sparsity and overlook the interactions among covariates. Here on the other hand, we devise a new regularization term in the Lasso regression model to impose high order interactions between covariates and responses. Specifically, we first construct a feature hypergraph to model the high-order relations among covariates, in which each node corresponds to a covariate and each hyperedge has a weight corresponding to the interaction information among covariates connected by that hyperedge. For the hyperedge weight, we use multidimensional interaction information (MII) to measure the significance of different covariate combinations with respect to response. Secondly, we use the feature hypergraph as a regularizer on the covariate coefficients which can automatically adjust the relevance measure between a covariate and the response by the interaction weights obtained from hypergraph. Finally, an efficient alternating direction method of multipliers (ADMM) is presented to solve the resulting sparse optimization problem. Extensive experiments on different data sets show that although our proposed model is not a convex problem, it outperforms both its approximately convex counterparts and a number of state-of-the-art feature selection methods.

High-order covariate interacted Lasso for feature selection.

Adaptive Lasso in high-dimensional settings

A Variance Minimization Criterion to Feature Selection Using Laplacian Regularization

Consistent group selection in high-dimensional linear regression

High-dimensional Interactions Detection with Sparse Principal Hessian Matrix

Interaction Pursuit with Feature Screening and Selection

Dual Lasso Selector

Longitudinal LASSO: Jointly Learning Features and Temporal Contingency for Outcome Prediction

Lasso with Latents: Efficient Estimation, Covariate Rescaling, and Computational-Statistical Gaps

Nonnegative Adaptive Lasso for Ultra-High Dimensional Regression Models and A Two-Stage Method Applied in Financial Modeling

A Sparse-Group Lasso

Sequential Lasso for feature selection with ultra-high dimensional feature space

Regularization Methods for High-Dimensional Instrumental Variables Regression With an Application to Genetical Genomics

Sparse Variable Selection on High Dimensional Heterogeneous Data With Tree Structured Responses

The Adaptive Lasso and Its Oracle Properties

A New Covariate Selection Strategy for High Dimensional Data in Causal Effect Estimation with Multivariate Treatments

Sequential profile Lasso for ultra-high-dimensional partially linear models

Asymptotic Properties of Lasso+mLS and Lasso+Ridge in Sparse High-Dimensional Linear Regression

Variable Selection for Generalized Linear Model with Highly Correlated Covariates

Adaptive Lasso for High Dimensional Regression and Gaussian Graphical Modeling

Robust Fused Lasso Penalized Huber Regression with Nonasymptotic Property and Implementation Studies