Abstract:Grouping structures arise naturally in many high dimensional statistical problems. Incorporation of grouping information can efficiently improve the statistical accuracy and model interpretability. In addition, nonnegative constraints are essential to cope with index tracking problems. This paper proposes the nonnegative hierarchical lasso with nonnegative constraints on the coefficients both in low dimensional setting and ultra high dimensional setting, which is capable of simultaneous selection at both the group and within-group levels with overlap, namely the bi-level selection. In theoretical analysis, we show the nonnegative hierarchical lasso enjoys the oracle properties in group selection when the number of covariates diverges with the sample size under certain regularity conditions. Since there are less works devoted to the theoretical properties of bi-level selection methods in cases where the number of variables or groups is much larger than the sample size, we also derive the oracle inequalities for the prediction and l(1) estimation errors of the estimator under the restricted eigen-value conditions on the design matrix. It is shown to have group selection consistency and estimation consistency in ultra high-dimensional sparse linear regression models. To get the solution of the nonnegative hierarchical lasso, we propose a fast and efficient iterative half thresholding-based local linear approximation algorithm (IHT-LLA) for solving. Simulations indicate that the nonnegative hierarchical lasso outperforms other nonnegative regularization methods and is robust against possible mis-specified grouping structure. Besides, we further apply our method to the index tracking problems.

Nonnegative hierarchical lasso with a mixed (1,1/2)-penalty and a fast solver

Adaptive Lasso in high-dimensional settings

Nonnegative Adaptive Lasso for Ultra-High Dimensional Regression Models and A Two-Stage Method Applied in Financial Modeling

Nonnegative-lasso and application in index tracking

Consistent group selection in high-dimensional linear regression

Overlapping Group Lasso for High-Dimensional Generalized Linear Models

Bi-level variable selection via adaptive sparse group Lasso

A sparse-group lasso

A High-dimensional M-estimator Framework for Bi-level Variable Selection

A Group Adaptive Elastic-Net Approach for Variable Selection in High-Dimensional Linear Regression

An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems

Robust group non-convex estimations for high-dimensional partially linear models

A first-order optimization algorithm for statistical learning with hierarchical sparsity structure

Exclusive Feature Learning on Arbitrary Structures via l(1,2)-norm

The Linearized Alternating Direction Method of Multipliers for Sparse Group LAD Model.

Nonnegative Sparse Group Lasso with an Application in Financial Index Tracking

The high-dimensional statistical analysis of sparse Group Lasso

Convergence and Sparsity of Lasso and Group Lasso in High-Dimensional Generalized Linear Models

Group Regularized Estimation under Structural Hierarchy

Non-asymptotic Oracle Inequalities for the Lasso and Group Lasso in high dimensional logistic model