Abstract:Variable selection is fundamental to high-dimensional statistical modeling, including nonparametric regression. Many approaches in use are stepwise selection procedures, which can be computationally expensive and ignore stochastic errors in the variable selection process. In this article, penalized likelihood approaches are proposed to handle these kinds of problems. The proposed methods select variables and estimate coefficients simultaneously. Hence they enable us to construct confidence intervals for estimated parameters. The proposed approaches are distinguished from others in that the penalty functions are symmetric, nonconcave on (0, ∞), and have singularities at the origin to produce sparse solutions. Furthermore, the penalty functions should be bounded by a constant to reduce bias and satisfy certain conditions to yield continuous solutions. A new algorithm is proposed for optimizing penalized likelihood functions. The proposed ideas are widely applicable. They are readily applied to a variety of parametric models such as generalized linear models and robust regression models. They can also be applied easily to nonparametric modeling by using wavelets and splines. Rates of convergence of the proposed penalized likelihood estimators are established. Furthermore, with proper choice of regularization parameters, we show that the proposed estimators perform as well as the oracle procedure in variable selection; namely, they work as well as if the correct submodel were known. Our simulation shows that the newly proposed methods compare favorably with other variable selection techniques. Furthermore, the standard error formulas are tested to be accurate enough for practical applications.

A globally convergent algorithm for lasso-penalized mixture of linear regression models

On Mixture Regression Shrinkage and Selection via the MR-LASSO

Model Selection for Exponential Power Mixture Regression Models

Simultaneous Estimation and Variable Selection in Median Regression Using Lasso-type Penalty

Proximal Iteration for Nonlinear Adaptive Lasso

Outlier detection and robust variable selection via the penalized weighted LAD-LASSO method

Variable Selection for Linear Mixed Effects Model Via Penalization Approaches

Non-Concave Penalization in Linear Mixed-Effects Models and Regularized Selection of Fixed Effects

Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties

A Fast Solution to the Lasso Problem with Equality Constraints

The linearized alternating direction method of multipliers for low-rank and fused LASSO matrix regression model

Least Squares Approximation with a Diverging Number of Parameters

Variable Selection Using Penalized Empirical Likelihood

Fast selection of nonlinear mixed effect models using penalized likelihood

Stability of a Generalized Debiased Lasso with Applications to Resampling-Based Variable Selection

Adaptive Penalized Likelihood method for Markov Chains

A descent algorithm for constrained LAD-Lasso estimation with applications in portfolio selection

Nonnegative Adaptive Lasso for Ultra-High Dimensional Regression Models and A Two-Stage Method Applied in Financial Modeling

A linearization procedure and a VDM/ECM algorithm for penalized and constrained nonparametric maximum likelihood estimation for mixture models

AN EXTENDED LINEARIZED ALTERNATING DIRECTION METHOD OF MULTIPLIERS FOR FUSED-LASSO PENALIZED LINEAR REGRESSION

Interacting Particle Langevin Algorithm for Maximum Marginal Likelihood Estimation