Abstract:Variable selection is fundamental to high-dimensional statistical modeling, including nonparametric regression. Many approaches in use are stepwise selection procedures, which can be computationally expensive and ignore stochastic errors in the variable selection process. In this article, penalized likelihood approaches are proposed to handle these kinds of problems. The proposed methods select variables and estimate coefficients simultaneously. Hence they enable us to construct confidence intervals for estimated parameters. The proposed approaches are distinguished from others in that the penalty functions are symmetric, nonconcave on (0, ∞), and have singularities at the origin to produce sparse solutions. Furthermore, the penalty functions should be bounded by a constant to reduce bias and satisfy certain conditions to yield continuous solutions. A new algorithm is proposed for optimizing penalized likelihood functions. The proposed ideas are widely applicable. They are readily applied to a variety of parametric models such as generalized linear models and robust regression models. They can also be applied easily to nonparametric modeling by using wavelets and splines. Rates of convergence of the proposed penalized likelihood estimators are established. Furthermore, with proper choice of regularization parameters, we show that the proposed estimators perform as well as the oracle procedure in variable selection; namely, they work as well as if the correct submodel were known. Our simulation shows that the newly proposed methods compare favorably with other variable selection techniques. Furthermore, the standard error formulas are tested to be accurate enough for practical applications.

Partial Penalized Likelihood Ratio Test under Sparse Case

Penalized Independence Rule for Testing High-Dimensional Hypotheses

Penalized Empirical Likelihood for High-Dimensional Partially Linear Varying Coefficient Model with Measurement Errors

Inference for possibly misspecified generalized linear models with nonpolynomial-dimensional nuisance parameters

A Regularization-Based Adaptive Test for High-Dimensional Generalized Linear Models

Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties

Testing covariates in high-dimensional regression

Strong Oracle Guarantees for Partial Penalized Tests of High Dimensional Generalized Linear Models

Estimation and Inference in Ultrahigh Dimensional Partially Linear Single-Index Models

Testing for the Presence of Significant Covariates Through Conditional Marginal Regression

Empirical Likelihood Test for Regression Coefficients in High Dimensional Partially Linear Models

Generalized Likelihood Ratio Test in Partially Linear Models

Post Selection Shrinkage Estimation for High Dimensional Data Analysis

Asymptotic Power of Likelihood Ratio Tests for High Dimensional Data

Adaptive L-statistics for high dimensional test problem

Computational and Numerical Properties of a Broadband Subspace-Based Likelihood Ratio Test

Testing generalized linear models with high-dimensional nuisance parameter.

Power Enhancement in High Dimensional Cross-Sectional Tests

Penalized Inverse Probability Weighted Estimators for Weighted Rank Regression with Missing Covariates

Weighted Lasso Estimates for Sparse Logistic Regression: Non-Asymptotic Properties with Measurement Errors

Penalized Sparse Covariance Regression with High Dimensional Covariates