Abstract:A sparse modeling is a major topic in machine learning and statistics. LASSO (Least Absolute Shrinkage and Selection Operator) is a popular sparse modeling method while it has been known to yield unexpected large bias especially at a sparse representation. There have been several studies for improving this problem such as the introduction of non-convex regularization terms. The important point is that this bias problem directly affects model selection in applications since a sparse representation cannot be selected by a prediction error based model selection even if it is a good representation. In this article, we considered to improve this problem by introducing a scaling that expands LASSO estimator to compensate excessive shrinkage, thus a large bias in LASSO estimator. We here gave an empirical value for the amount of scaling. There are two advantages of this scaling method as follows. Since the proposed scaling value is calculated by using LASSO estimator, we only need LASSO estimator that is obtained by a fast and stable optimization procedure such as LARS (Least Angle Regression) under LASSO modification or coordinate descent. And, the simplicity of our scaling method enables us to derive SURE (Stein's Unbiased Risk Estimate) under the modified LASSO estimator with scaling. Our scaling method together with model selection based on SURE is fully empirical and do not need additional hyper-parameters. In a simple numerical example, we verified that our scaling method actually improves LASSO and the SURE based model selection criterion can stably choose an appropriate sparse model.

A new method for penalized least squares by weight scaling factor

A Method for Dynamic Locally Weighting Based on Sequential Quadratic Programming

Improved Weighted Least Squares Support Vector Machines Algorithm And Its Applications In Spectroscopic Quantitative Analysis

Weighted least squares and quasi-weighted least squares combined method for data rectification

A Note on the Optimal Smoothing in Partial Linear Models with Penalized Least Squares Estimator

Weighted least squares regression with the best robustness and high computability

A compromise criterion for weighted least squares estimates

Kernel density weighted method of obtaining the least square support vector machines weightes

A new penalized least absolute deviation model for high dimensional sparse linear regression and an efficient sequential linear programming algorithm

Sparse Poisson Regression with Penalized Weighted Score Function

A New Penalized Recursive Least Squares Method With a Variable Regularization Factor for Adaptive Sparse Filtering.

On the Balancing Principle for Some Problems of Numerical Analysis.

Sequential Quadratic Programming Method for Nonlinear Least Squares Estimation and Its Application

Calibrating Least Squares Semidefinite Programming with Equality and Inequality Constraints

Semi-Smooth Newton Algorithm for Non-Convex Penalized Linear Regression

A Performance Nondestructive Pruning Method For Reduced Robust Weighted Least Squares Support Vector Regression

A Smoothing Method of the Square-root Exact Penalty Function for Inequality Constrained Optimization

On the Weighting Method for Mixed Least Squares–total Least Squares Problems

On an improvement of LASSO by scaling

A Family of Scaled Factorized Broyden-Like Methods for Nonlinear Least Squares Problems

On Smoothing Exact Penalty Function for Nonlinear Constrained Optimization Problems