Abstract:Preconditioning is a technique from numerical linear algebra that can accelerate algorithms to solve systems of equations. In this paper, we demonstrate how preconditioning can circumvent a stringent assumption for sign consistency in sparse linear regression. Given X ∈ R^n × p and Y ∈ R^n that satisfy the standard regression equation, this paper demonstrates that even if the design matrix X does not satisfy the irrepresentable condition for the Lasso, the design matrix F X often does, where F ∈ R^n× n is a preconditioning matrix defined in this paper. By computing the Lasso on (F X, F Y), instead of on (X, Y), the necessary assumptions on X become much less stringent. Our preconditioner F ensures that the singular values of the design matrix are either zero or one. When n≥ p, the columns of F X are orthogonal and the preconditioner always circumvents the stringent assumptions. When p≥ n, F projects the design matrix onto the Stiefel manifold; the rows of F X are orthogonal. We give both theoretical results and simulation results to show that, in the high dimensional case, the preconditioner helps to circumvent the stringent assumptions, improving the statistical performance of a broad class of model selection techniques in linear regression. Simulation results are particularly promising.

Preconditioned Matrix and Its Structure Methods

Preconditioning Technique for Symmetric M-matrices

A Note on Preconditioning for M-Matrix

Structure Preprocessing Method for the System of Unclosed Linear Algebraic Equations

A preconditioned version of the MBP iteration method for a class of complex symmetric linear systems

A preconditioned iteration method for solving saddle point problems

Preconditioning techniques for generalized Sylvester matrix equations

A Generalization of T. Chan’s Preconditioner

A Preconditioned MINRES Method for Block Lower Triangular Toeplitz Systems

Preconditioned AOR Iterative Method for H-matrices

Practical Restrictively Preconditioned Conjugate Gradient Methods for a Class of Block Two-by-Two Linear Systems

A preconditioner for solving linear programming problems with dense columns

Learning Preconditioners for Conjugate Gradient PDE Solvers

A CONSTRAINT PRECONDITIONER FOR SOLVING SYMMETRIC POSITIVE DEFINITE SYSTEMS AND APPLICATION TO THE HELMHOLTZ EQUATIONS AND POISSON EQUATIONS

A Preconditioner based on Non-uniform Row Sampling for Linear Least Squares Problems

New choices of preconditioning matrices for generalized inexact parameterized iterative methods

Preconditioned iterative method for nonsymmetric saddle point linear systems

Preconditioning to Comply with the Irrepresentable Condition

Polynomial Preconditioning for the Action of the Matrix Square Root and Inverse Square Root

A Modified SSOR-like Preconditioner for Non-Hermitian Positive Definite Matrices