Abstract:It is difficult to find the optimal sparse solution of a manifold learning based dimensionality reduction algorithm. The lasso or the elastic net penalized manifold learning based dimensionality reduction is not directly a lasso penalized least square problem and thus the least angle regression (LARS) (Efron et al., Ann Stat 32(2):407---499, 2004), one of the most popular algorithms in sparse learning, cannot be applied. Therefore, most current approaches take indirect ways or have strict settings, which can be inconvenient for applications. In this paper, we proposed the manifold elastic net or MEN for short. MEN incorporates the merits of both the manifold learning based dimensionality reduction and the sparse learning based dimensionality reduction. By using a series of equivalent transformations, we show MEN is equivalent to the lasso penalized least square problem and thus LARS is adopted to obtain the optimal sparse solution of MEN. In particular, MEN has the following advantages for subsequent classification: (1) the local geometry of samples is well preserved for low dimensional data representation, (2) both the margin maximization and the classification error minimization are considered for sparse projection calculation, (3) the projection matrix of MEN improves the parsimony in computation, (4) the elastic net penalty reduces the over-fitting problem, and (5) the projection matrix of MEN can be interpreted psychologically and physiologically. Experimental evidence on face recognition over various popular datasets suggests that MEN is superior to top level dimensionality reduction algorithms.

Stability of the Elastic Net Estimator

Average case analysis of Lasso under ultra-sparse conditions

Weighted LASSO for Sparse Recovery with Statistical Prior Support Information

Elastic-Net Regularization For Low-Rank Matrix Recovery

Iterative Reweighted Framework Based Algorithms for Sparse Linear Regression with Generalized Elastic Net Penalty

Group Sparse Recovery Via Group Square-Root Elastic Net and the Iterative Multivariate Thresholding-Based Algorithm

Robust Elastic Net Regression

On Estimation Error Bounds of the Elastic Net when P ≫ N

Enmsp: an elastic-net multi-step screening procedure for high-dimensional regression

Elastic Net Procedure for Partially Linear Models

Rates of Convergence of the Adaptive Elastic Net and the Post-Selection Procedure in Ultra-High Dimensional Sparse Models

Regularity Properties for Sparse Regression

Adaptive Elastic Net and Separate Selection from Least Squares for Ultra-High Dimensional Regression Models

On Sparse Vector Recovery Performance in Structurally Orthogonal Matrices via LASSO

A Group Adaptive Elastic-Net Approach for Variable Selection in High-Dimensional Linear Regression

A Robust Variable Selection Method for Sparse Online Regression Via the Elastic Net Penalty

HIGH-DIMENSIONAL LEAST SQUARE MATRIX REGRESSION VIA ELASTIC NET PENALTY

Manifold Elastic Net: a Unified Framework for Sparse Dimension Reduction

Sparse Nonlinear Regression: Parameter Estimation and Asymptotic Inference

Elastic-net Regularized High-dimensional Negative Binomial Regression: Consistency and Weak Signals Detection

Properties and Iterative Methods for the Elastic Net with ℓp-Norm Errors