Low solution rank of the matrix LASSO under RIP with consequences for rank-constrained algorithms

Andrew D. McRae

2024-04-19

Abstract:We show that solutions to the popular convex matrix LASSO problem (nuclear-norm--penalized linear least-squares) have low rank under similar assumptions as required by classical low-rank matrix sensing error bounds. Although the purpose of the nuclear norm penalty is to promote low solution rank, a proof has not yet (to our knowledge) been provided outside very specific circumstances. Furthermore, we show that this result has significant theoretical consequences for nonconvex rank-constrained optimization approaches. Specifically, we show that if (a) the ground truth matrix has low rank, (b) the (linear) measurement operator has the matrix restricted isometry property (RIP), and (c) the measurement error is small enough relative to the nuclear norm penalty, then the (unique) LASSO solution has rank (approximately) bounded by that of the ground truth. From this, we show (a) that a low-rank--projected proximal gradient descent algorithm will converge linearly to the LASSO solution from any initialization, and (b) that the nonconvex landscape of the low-rank Burer-Monteiro--factored problem formulation is benign in the sense that all second-order critical points are globally optimal and yield the LASSO solution.

Optimization and Control,Statistics Theory

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

The main purpose of this paper is to demonstrate that the solution to the matrix LASSO problem (i.e., the linear least squares problem with nuclear norm regularization) exhibits low-rank properties under certain conditions. Specifically:

1. **Low-Rank Guarantee**: The paper shows that when the true matrix has low rank, the measurement operator has the matrix restricted isometry property (RIP), and the measurement error is sufficiently small, the solution to the matrix LASSO problem will also have low rank. Although the purpose of nuclear norm regularization is to promote low-rank solutions, there was previously no theoretical proof for this in general cases.

2. **Theoretical Results for Non-Convex Optimization Algorithms**:
- The paper further demonstrates the significance of this result for non-convex low-rank constrained optimization methods. In particular, if the solution has low rank, the low-rank projected gradient descent algorithm will converge linearly to the LASSO solution from any initial value.
- Additionally, for problems in the low-rank Burer-Monteiro decomposition form, all second-order critical points are global optima.

Through these results, the paper not only provides a theoretical guarantee for the low-rank property of the matrix LASSO problem solution but also offers a theoretical foundation for the effectiveness of non-convex optimization algorithms.