Abstract:Grouping structures arise naturally in many high dimensional statistical problems. Incorporation of grouping information can efficiently improve the statistical accuracy and model interpretability. In addition, nonnegative constraints are essential to cope with index tracking problems. This paper proposes the nonnegative hierarchical lasso with nonnegative constraints on the coefficients both in low dimensional setting and ultra high dimensional setting, which is capable of simultaneous selection at both the group and within-group levels with overlap, namely the bi-level selection. In theoretical analysis, we show the nonnegative hierarchical lasso enjoys the oracle properties in group selection when the number of covariates diverges with the sample size under certain regularity conditions. Since there are less works devoted to the theoretical properties of bi-level selection methods in cases where the number of variables or groups is much larger than the sample size, we also derive the oracle inequalities for the prediction and l(1) estimation errors of the estimator under the restricted eigen-value conditions on the design matrix. It is shown to have group selection consistency and estimation consistency in ultra high-dimensional sparse linear regression models. To get the solution of the nonnegative hierarchical lasso, we propose a fast and efficient iterative half thresholding-based local linear approximation algorithm (IHT-LLA) for solving. Simulations indicate that the nonnegative hierarchical lasso outperforms other nonnegative regularization methods and is robust against possible mis-specified grouping structure. Besides, we further apply our method to the index tracking problems.

Exclusive Feature Learning on Arbitrary Structures via l(1,2)-norm

Exclusive Group Lasso for Structured Variable Selection

Sparse Feature Grouping Based on $\ell_{1/2}$ Norm Regularization

Feature selection guided by structural information

Two-Layer Feature Reduction for Sparse-Group Lasso Via Decomposition of Convex Sets.

Unsupervised Feature Selection with Constrained ℓ₂,₀-Norm and Optimized Graph

Local Sparse Discriminative Feature Selection

Nonnegative hierarchical lasso with a mixed (1,1/2)-penalty and a fast solver

Structured Learning for Unsupervised Feature Selection with High-Order Matrix Factorization

Exclusive Sparsity Norm Minimization with Random Groups Via Cone Projection

Outliers Robust Unsupervised Feature Selection for Structured Sparse Subspace

A sparse-group lasso

Outlier-Robust Feature Selection with ℓ<inf>2,1</inf>-Norm Minimization and Group Row-Sparsity Induced Constraints

Discriminative Feature Selection Via A Structured Sparse Subspace Learning Module

Faster Training Algorithms for Structured Sparsity-Inducing Norm

An Ideal Local Structure Learning for Unsupervised Feature Selection

Exact Top-k Feature Selection via l2, 0-Norm Constraint.

Efficient Feature Selection via $\ell _{2, 0}$ℓ2, 0-norm Constrained Sparse Regression.

Double-Structured Sparsity Guided Flexible Embedding Learning for Unsupervised Feature Selection

Nonlinear Feature Selection Neural Network Via Structured Sparse Regularization