Abstract:Variable selection is fundamental to high-dimensional statistical modeling, including nonparametric regression. Many approaches in use are stepwise selection procedures, which can be computationally expensive and ignore stochastic errors in the variable selection process. In this article, penalized likelihood approaches are proposed to handle these kinds of problems. The proposed methods select variables and estimate coefficients simultaneously. Hence they enable us to construct confidence intervals for estimated parameters. The proposed approaches are distinguished from others in that the penalty functions are symmetric, nonconcave on (0, ∞), and have singularities at the origin to produce sparse solutions. Furthermore, the penalty functions should be bounded by a constant to reduce bias and satisfy certain conditions to yield continuous solutions. A new algorithm is proposed for optimizing penalized likelihood functions. The proposed ideas are widely applicable. They are readily applied to a variety of parametric models such as generalized linear models and robust regression models. They can also be applied easily to nonparametric modeling by using wavelets and splines. Rates of convergence of the proposed penalized likelihood estimators are established. Furthermore, with proper choice of regularization parameters, we show that the proposed estimators perform as well as the oracle procedure in variable selection; namely, they work as well as if the correct submodel were known. Our simulation shows that the newly proposed methods compare favorably with other variable selection techniques. Furthermore, the standard error formulas are tested to be accurate enough for practical applications.

Consistent variable selection via the optimal discovery procedure in multiple testing

A one-covariate-at-a-time multiple testing approach to variable selection in additive models

Variable Selection in High-Dimensional Error-in-Variables Models Via Controlling the False Discovery Proportion

Faithful Variable Screening for High-Dimensional Convex Regression

E M ] 2 6 A pr 2 02 2 A One-Covariate-ata-Time Multiple Testing Approach to Variable Selection in Additive Models ∗

A One-Covariate-at-a-Time Method for Nonparametric Additive Models

Variable Selection for High Dimensional Gaussian Copula Regression Model: an Adaptive Hypothesis Testing Procedure.

Variable Selection via Adaptive False Negative Control in Linear Regression

A Unified Variable Selection Approach for Varying Coefficient Models

New Algorithms and Software for Significance Controlled Variable Selection

A Transparent and Nonlinear Method for Variable Selection

False discovery control for penalized variable selections with high-dimensional covariates.

Variable selection and estimation for semi-parametric multiple-index models

A New Multiple Testing Method in the Dependent Case

Variable selection with error control: Another look at Stability Selection

Robust Variable and Interaction Selection for Logistic Regression and Multiple Index Models

Outlier detection and robust variable selection via the penalized weighted LAD-LASSO method

Optimal False Discovery Rate Control for Large Scale Multiple Testing with Auxiliary Information

A Novel Approach for Fast Detection of Multiple Change Points in Linear Models

A Family of Bayes Multiple Testing Procedures

Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties