Abstract:Scaled sparse linear regression jointly estimates the regression coefficients and noise level in a linear model. It chooses an equilibrium with a sparse regression method by iteratively estimating the noise level via the mean residual square and scaling the penalty in proportion to the estimated noise level. The iterative algorithm costs little beyond the computation of a path or grid of the sparse regression estimator for penalty levels above a proper threshold. For the scaled lasso, the algorithm is a gradient descent in a convex minimization of a penalized joint loss function for the regression coefficients and noise level. Under mild regularity conditions, we prove that the scaled lasso simultaneously yields an estimator for the noise level and an estimated coefficient vector satisfying certain oracle inequalities for prediction, the estimation of the noise level and the regression coefficients. These inequalities provide sufficient conditions for the consistency and asymptotic normality of the noise-level estimator, including certain cases where the number of variables is of greater order than the sample size. Parallel results are provided for least-squares estimation after model selection by the scaled lasso. Numerical results demonstrate the superior performance of the proposed methods over an earlier proposal of joint convex minimization. Copyright 2012, Oxford University Press.

A Note on the Scaled Total Least Squares Problem

An Algorithm for Solving Scaled Total Least Squares Problems.

An algorithm for solving rank-deficient scaled total least square problems

A Note on the Sensitivity of the Solution of the Weighted Linear Least Squares Problem

Condition Numbers of Multidimensional Mixed Least Squares-Total Least Squares Problems

A contribution to perturbation analysis for total least squares problems

On the Weighting Method for Mixed Least Squares–total Least Squares Problems

Perturbation Analysis and Randomized Algorithms for Large-Scale Total Least Squares Problems

Randomized algorithms for total least squares problems.

Monotonic Properties of the Least Squares Mean

Perturbation bounds for constrained and weighted least squares problems

Condition Numbers of the Multidimensional Total Least Squares Problems Having More Than One Solution

On the Condition Number of the Total Least Squares Problem

The QLY Least-Squares and the QLY Least-Squares Minimal-Norm of Linear Dual Least Squares Problems

A contribution to the condition number of the total least squares problem

Scaled sparse linear regression

Condition Numbers for Structured Least Squares Problems

On the partial condition numbers for the indefinite least squares problem

Regularized Total Least Squares: Computational Aspects and Error Bounds.

Small Sample Statistical Condition Estimation for the Total Least Squares Problem

Total-variation minimization with bound constraints