Abstract:Variable selection is fundamental to high-dimensional statistical modeling, including nonparametric regression. Many approaches in use are stepwise selection procedures, which can be computationally expensive and ignore stochastic errors in the variable selection process. In this article, penalized likelihood approaches are proposed to handle these kinds of problems. The proposed methods select variables and estimate coefficients simultaneously. Hence they enable us to construct confidence intervals for estimated parameters. The proposed approaches are distinguished from others in that the penalty functions are symmetric, nonconcave on (0, ∞), and have singularities at the origin to produce sparse solutions. Furthermore, the penalty functions should be bounded by a constant to reduce bias and satisfy certain conditions to yield continuous solutions. A new algorithm is proposed for optimizing penalized likelihood functions. The proposed ideas are widely applicable. They are readily applied to a variety of parametric models such as generalized linear models and robust regression models. They can also be applied easily to nonparametric modeling by using wavelets and splines. Rates of convergence of the proposed penalized likelihood estimators are established. Furthermore, with proper choice of regularization parameters, we show that the proposed estimators perform as well as the oracle procedure in variable selection; namely, they work as well as if the correct submodel were known. Our simulation shows that the newly proposed methods compare favorably with other variable selection techniques. Furthermore, the standard error formulas are tested to be accurate enough for practical applications.

Variable Selection in Expectile Regression

Expectile regression for analyzing heteroscedasticity in high dimension

Semiparametric Expectile Regression for High-dimensional Heavy-tailed and Heterogeneous Data

Robust Estimation and Shrinkage in Ultrahigh Dimensional Expectile Regression with Heavy Tails and Variance Heterogeneity

Conditional Tail-Related Risk Estimation Using Composite Asymmetric Least Squares and Empirical Likelihood

An Improved Algorithm for High-Dimensional Continuous Threshold Expectile Model with Variance Heterogeneity

Variable Selection Using Penalized Empirical Likelihood

Variable Selection for Generalized Varying Coefficient Partially Linear Models with Diverging Number of Parameters

Variable Selection for the Partial Linear Single-Index Model

Inference for High-Dimensional Linear Expectile Regression with De-Biasing Method

Variable Screening and Model Averaging for Expectile Regressions

Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties

Variable Selection with Exponential Weights and $l_0$-Penalization

Robust exponential squared loss-based variable selection for high-dimensional single-index varying-coefficient model

Distributed optimization and statistical learning for large-scale penalized expectile regression

Variable Selection for Linear Mixed Effects Model Via Penalization Approaches

Post Selection Shrinkage Estimation for High Dimensional Data Analysis

Penalized Maximum Tangent Likelihood Estimation and Robust Variable Selection

Simultaneous Estimation and Variable Selection in Median Regression Using Lasso-type Penalty

Variable Selection Via Generalized SELO-Penalized Cox Regression Models

Efficient penalized estimating method in the partially varying-coefficient single-index model