Abstract:Variable selection is fundamental to high-dimensional statistical modeling, including nonparametric regression. Many approaches in use are stepwise selection procedures, which can be computationally expensive and ignore stochastic errors in the variable selection process. In this article, penalized likelihood approaches are proposed to handle these kinds of problems. The proposed methods select variables and estimate coefficients simultaneously. Hence they enable us to construct confidence intervals for estimated parameters. The proposed approaches are distinguished from others in that the penalty functions are symmetric, nonconcave on (0, ∞), and have singularities at the origin to produce sparse solutions. Furthermore, the penalty functions should be bounded by a constant to reduce bias and satisfy certain conditions to yield continuous solutions. A new algorithm is proposed for optimizing penalized likelihood functions. The proposed ideas are widely applicable. They are readily applied to a variety of parametric models such as generalized linear models and robust regression models. They can also be applied easily to nonparametric modeling by using wavelets and splines. Rates of convergence of the proposed penalized likelihood estimators are established. Furthermore, with proper choice of regularization parameters, we show that the proposed estimators perform as well as the oracle procedure in variable selection; namely, they work as well as if the correct submodel were known. Our simulation shows that the newly proposed methods compare favorably with other variable selection techniques. Furthermore, the standard error formulas are tested to be accurate enough for practical applications.

Consistent Variable Selection for a Nonparametric Nonlinear System by Inverse and Contour Regressions.

Identification of nonlinearly parameterized nonlinear models: application to mass balance systems.

Variable Selection and Identification of High-Dimensional Nonparametric Nonlinear Systems by Directional Regression

A Transparent and Nonlinear Method for Variable Selection

Faithful Variable Screening for High-Dimensional Convex Regression

Variable Selection for Generalized Varying Coefficient Partially Linear Models with Diverging Number of Parameters

Sufficient Variable Selection of High Dimensional Nonparametric Nonlinear Systems Based on Fourier Spectrum of Density-Weighted Derivative

Nonparametric Variable Selection and Its Application to Additive Models

Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties

A Unified Variable Selection Approach for Varying Coefficient Models

Robust Variable Selection for Nonlinear Models with Diverging Number of Parameters

EFFICIENT KERNEL-BASED VARIABLE SELECTION WITH SPARSISTENCY

Towards a Unified Framework for Uncertainty-aware Nonlinear Variable Selection with Theoretical Guarantees

Robust Variable Selection for Single-Index Varying-Coefficient Model with Missing Data in Covariates

Variable selection in semiparametric regression modeling

Robust Variable Projection Algorithm for the Identification of Separable Nonlinear Models

Variable Selection for High Dimensional Partially Linear Varying Coefficient Errors-in-variables Models

Variable Selection for Multivariate Functional Data Via Conditional Correlation Learning

Variable Selection in a Class of Single-Index Models

Robust Variable Selection and Parametric Component Identification in Varying Coefficient Models

Consistent Tuning Parameter Selection in High Dimensional Sparse Linear Regression