Abstract:Both the smoothly clipped absolute deviation (SCAD) and the minimax concave penalty (MCP) penalized linear regression models are capable of dealing with variable selection and parameter estimation simultaneously. Theoretically, these two models enjoy the oracle property even in the high dimensional settings where the number of predictors $p$ may be much larger than the number of observations $n$. However, numerically, it is quite challenging to develop fast and stable algorithms due to their non-convexity and non-smoothness. In this paper we develop a fast algorithm for SCAD and MCP penalized problems. First, we derive that the global minimizers of both models are roots of some nonsmooth equations. Then, Semi-smooth Newton (SSN) algorithm is employed to solve the equations. We prove the SSN algorithm converges locally and superlinearly to KKT points. Computational complexity analysis demonstrates that the cost of SSN algorithm per iteration is $O(np)$. Combining with the warmstarting technique SSN algorithm can be very efficient. Simulation studies and real data examples show that the SSN algorithm outperforms coordinate descent and DC proximal Newton algorithms in computational efficiency while reaching comparable accuracy.

Globally consistent model selection in semi-parametric additive coefficient models

Variable Selection for Generalized Varying Coefficient Partially Linear Models with Diverging Number of Parameters

PENALIZED VARIABLE SELECTION PROCEDURE FOR COX MODELS WITH SEMIPARAMETRIC RELATIVE RISK

Estimating Functions and Derivatives Via Adaptive Penalized Splines.

ADDITIVE COEFFICIENT MODELING VIA POLYNOMIAL SPLINE

Variable Selection for the Partial Linear Single-Index Model

Semi-Smooth Newton Algorithm for Non-Convex Penalized Linear Regression

Non-convex Penalized Estimation in High-Dimensional Models with Single-Index Structure

Variable selection for semiparametric varying-coefficient spatial autoregressive models with a diverging number of parameters

Sparse and Efficient Estimation for Partial Spline Models with Increasing Dimension

Robust Variable Selection and Parametric Component Identification in Varying Coefficient Models

Variable Selection Via Generalized SELO-Penalized Cox Regression Models

Efficient penalized estimating method in the partially varying-coefficient single-index model

Variable Selection for Partially Time-Varying Coefficient Error-In-Variables Models

Estimation and Inference for Varying-Coefficient Models with Nonstationary Regressors Using Penalized Splines

A New Variable Selection Approach for Varying Coefficient Models

Tuning parameter selector for the penalized likelihood method in multivariate generalized linear models

Variable selection via combined penalization in joint mean and variance models

Variable selection for partially linear single-index varying-coefficient model

Efficient estimation of a linear transformation model for current status data via penalized splines

A robust and efficient estimation and variable selection method for partially linear single-index models.