Abstract:Grouping structures arise naturally in many high dimensional statistical problems. Incorporation of grouping information can efficiently improve the statistical accuracy and model interpretability. In addition, nonnegative constraints are essential to cope with index tracking problems. This paper proposes the nonnegative hierarchical lasso with nonnegative constraints on the coefficients both in low dimensional setting and ultra high dimensional setting, which is capable of simultaneous selection at both the group and within-group levels with overlap, namely the bi-level selection. In theoretical analysis, we show the nonnegative hierarchical lasso enjoys the oracle properties in group selection when the number of covariates diverges with the sample size under certain regularity conditions. Since there are less works devoted to the theoretical properties of bi-level selection methods in cases where the number of variables or groups is much larger than the sample size, we also derive the oracle inequalities for the prediction and l(1) estimation errors of the estimator under the restricted eigen-value conditions on the design matrix. It is shown to have group selection consistency and estimation consistency in ultra high-dimensional sparse linear regression models. To get the solution of the nonnegative hierarchical lasso, we propose a fast and efficient iterative half thresholding-based local linear approximation algorithm (IHT-LLA) for solving. Simulations indicate that the nonnegative hierarchical lasso outperforms other nonnegative regularization methods and is robust against possible mis-specified grouping structure. Besides, we further apply our method to the index tracking problems.

The high-dimensional statistical analysis of sparse Group Lasso

Average case analysis of Lasso under ultra-sparse conditions

Adaptive Lasso in high-dimensional settings

A note on the group lasso and a sparse group lasso

A Sparse-Group Lasso

An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems

Group sparse optimization via l

Group Sparse Recovery Via the $\ell ^0(\ell ^2)$ Penalty: Theory and Algorithm

Overlapping Group Lasso for High-Dimensional Generalized Linear Models

Nonnegative hierarchical lasso with a mixed (1,1/2)-penalty and a fast solver

Non-asymptotic Oracle Inequalities for the Lasso and Group Lasso in high dimensional logistic model

Consistent group selection in high-dimensional linear regression

Change-point Estimation in High Dimensional Linear Regression Models Via Sparse Group Lasso

New Error Analysis for Lasso

The ℓ2, Regularized Group Sparse Optimization: Lower Bound Theory, Recovery Bound and Algorithms

Sparse Estimation Via Lower-Order Penalty Optimization Methods in High-Dimensional Linear Regression.

Error Bounds for Generalized Group Sparsity

High-dimensional generalized linear models and the lasso

Confidence Intervals for Sparse Precision Matrix Estimation Via Lasso Penalized D-trace Loss

Group Sparse Optimization Via Lp, Q Regularization.

High Dimensional Discriminant Analysis under Weak Sparsity