Abstract:We propose MC+, a fast, continuous, nearly unbiased and accurate method of penalized variable selection in high-dimensional linear regression. The LASSO is fast and continuous, but biased. The bias of the LASSO may prevent consistent variable selection. Subset selection is unbiased but computationally costly. The MC+ has two elements: a minimax concave penalty (MCP) and a penalized linear unbiased selection (PLUS) algorithm. The MCP provides the convexity of the penalized loss in sparse regions to the greatest extent given certain thresholds for variable selection and unbiasedness. The PLUS computes multiple exact local minimizers of a possibly nonconvex penalized loss function in a certain main branch of the graph of critical points of the penalized loss. Its output is a continuous piecewise linear path encompassing from the origin for infinite penalty to a least squares solution for zero penalty. We prove that at a universal penalty level, the MC+ has high probability of matching the signs of the unknowns, and thus correct selection, without assuming the strong irrepresentable condition required by the LASSO. This selection consistency applies to the case of $p\gg n$, and is proved to hold for exactly the MC+ solution among possibly many local minimizers. We prove that the MC+ attains certain minimax convergence rates in probability for the estimation of regression coefficients in $\ell_r$ balls. We use the SURE method to derive degrees of freedom and $C_p$-type risk estimates for general penalized LSE, including the LASSO and MC+ estimators, and prove their unbiasedness. Based on the estimated degrees of freedom, we propose an estimator of the noise level for proper choice of the penalty level.

Non-convex Penalized Estimation in High-Dimensional Models with Single-Index Structure

Non-convex penalized estimation in high-dimensional models with single-index structure

Nonconcave Penalized Inverse Regression in Single-Index Models with High Dimensional Predictors

Nonnegative Estimation and Variable Selection under Minimax Concave Penalty for Sparse High-Dimensional Linear Regression Models

Penalized Least Squares for Single Index Models

Variable Selection for the Partial Linear Single-Index Model

Semi-Smooth Newton Algorithm for Non-Convex Penalized Linear Regression

Variable selection and estimation for semi-parametric multiple-index models

NONCONCAVE PENALIZED M-ESTIMATION WITH A DIVERGING NUMBER OF PARAMETERS

Robust Variable Selection Based on Penalized Composite Quantile Regression for High-Dimensional Single-Index Models

Nearly unbiased variable selection under minimax concave penalty

A Semismooth Newton Algorithm for High-Dimensional Nonconvex Sparse Learning

PENALIZED VARIABLE SELECTION PROCEDURE FOR COX MODELS WITH SEMIPARAMETRIC RELATIVE RISK

Robust exponential squared loss-based variable selection for high-dimensional single-index varying-coefficient model

Efficient penalized estimating method in the partially varying-coefficient single-index model

A robust and efficient estimation and variable selection method for partially linear single-index models.

Quadratic Approximation via the SCAD Penalty with a Diverging Number of Parameters

SCAD-Penalized Least Absolute Deviation Regression in High-Dimensional Models

Variable selection for partially linear single-index varying-coefficient model

A robust and efficient estimation and variable selection method for partially linear models with large-dimensional covariates

Semiparametric Estimation And Variable Selection For Sparse Single Index Models In Increasing Dimension