Abstract:Grouping structures arise naturally in many high dimensional statistical problems. Incorporation of grouping information can efficiently improve the statistical accuracy and model interpretability. In addition, nonnegative constraints are essential to cope with index tracking problems. This paper proposes the nonnegative hierarchical lasso with nonnegative constraints on the coefficients both in low dimensional setting and ultra high dimensional setting, which is capable of simultaneous selection at both the group and within-group levels with overlap, namely the bi-level selection. In theoretical analysis, we show the nonnegative hierarchical lasso enjoys the oracle properties in group selection when the number of covariates diverges with the sample size under certain regularity conditions. Since there are less works devoted to the theoretical properties of bi-level selection methods in cases where the number of variables or groups is much larger than the sample size, we also derive the oracle inequalities for the prediction and l(1) estimation errors of the estimator under the restricted eigen-value conditions on the design matrix. It is shown to have group selection consistency and estimation consistency in ultra high-dimensional sparse linear regression models. To get the solution of the nonnegative hierarchical lasso, we propose a fast and efficient iterative half thresholding-based local linear approximation algorithm (IHT-LLA) for solving. Simulations indicate that the nonnegative hierarchical lasso outperforms other nonnegative regularization methods and is robust against possible mis-specified grouping structure. Besides, we further apply our method to the index tracking problems.

A Fast Solution to the Lasso Problem with Equality Constraints

Adaptive Lasso in high-dimensional settings

A cost-sensitive constrained Lasso

A Fast Method for Lasso and Logistic Lasso

Lasso with Latents: Efficient Estimation, Covariate Rescaling, and Computational-Statistical Gaps

Relaxed Adaptive Lasso and Its Asymptotic Results

A Fast and Scalable Pathwise-Solver for Group Lasso and Elastic Net Penalized Regression via Block-Coordinate Descent

A Greedy Homotopy Method for Regression with Nonconvex Constraints

A globally convergent algorithm for lasso-penalized mixture of linear regression models

Efficient sparse Hessian based algorithms for the clustered lasso problem.

Efficient Lasso Training from a Geometrical Perspective.

An Efficient Algorithm for a Class of Fused Lasso Problems

Multi-block linearized alternating direction method for sparse fused Lasso modeling problems

The Dual and Degrees of Freedom of Linearly Constrained Generalized Lasso

Fast Exact Conformalization of Lasso using Piecewise Linear Homotopy

New Error Analysis for Lasso

Sparsity and smoothness via the fused lasso

Proximal Iteration for Nonlinear Adaptive Lasso

Nonnegative hierarchical lasso with a mixed (1,1/2)-penalty and a fast solver

Safe Feature Elimination for the LASSO and Sparse Supervised Learning Problems

A fast and scalable framework for large-scale and ultrahigh-dimensional sparse regression with application to the UK Biobank