Abstract:Most existing methodologies of estimating low-rank matrices rely on Burer-Monteiro factorization, but these approaches can suffer from slow convergence, especially when dealing with solutions characterized by a large condition number, defined by the ratio of the largest to the $r$-th singular values, where $r$ is the search rank. While methods such as Scaled Gradient Descent have been proposed to address this issue, such methods are more complicated and sometimes have weaker theoretical guarantees, for example, in the rank-deficient setting. In contrast, this paper demonstrates the effectiveness of the projected gradient descent algorithm. Firstly, its local convergence rate is independent of the condition number. Secondly, under conditions where the objective function is rank-$2r$ restricted $L$-smooth and $\mu$-strongly convex, with $L/\mu < 3$, projected gradient descent with appropriate step size converges linearly to the solution. Moreover, a perturbed version of this algorithm effectively navigates away from saddle points, converging to an approximate solution or a second-order local minimizer across a wide range of step sizes. Furthermore, we establish that there are no spurious local minimizers in estimating asymmetric low-rank matrices when the objective function satisfies $L/\mu<3.$

What problem does this paper attempt to address?

### The Problem This Paper Attempts to Solve This paper primarily aims to address issues related to the convergence rate and condition number in low-rank matrix estimation. Specifically: 1. **Background and Limitations of Existing Methods**: - Most existing low-rank matrix estimation methods rely on Burer-Monteiro factorization, but these methods encounter slow convergence when dealing with large condition numbers. - Although some improved methods like Scaled Gradient Descent (Scaled GD) have been proposed to address this issue, they sometimes have weaker theoretical guarantees, especially in the case of rank deficiency. 2. **Main Contributions of the Research**: - The paper demonstrates the effectiveness of the Projected Gradient Descent algorithm (ProjGD), which has the following features: - **Local Convergence Rate**: The local convergence rate of ProjGD is unaffected by the condition number. - **Global Convergence Rate**: When the objective function satisfies certain specific conditions (such as rank-2r restricted smoothness and strong convexity), ProjGD can converge linearly to the solution, and the convergence rate is independent of the condition number. - **Saddle Point Navigation Ability**: The paper also proposes a perturbed version of ProjGD (PprojGD), which can effectively avoid saddle points and converge to an approximate solution or a second-order local minimum. 3. **Comparison of Theoretical Results**: - Compared to existing methods, ProjGD excels in the following aspects: - The local convergence rate is unaffected by the condition number, whereas other methods like Factorized Gradient Descent (FGD) have slower convergence rates when the condition number is large. - The global convergence analysis allows for a larger L/μ range and provides greater flexibility in step size selection. - The perturbed version PprojGD can better handle unknown parameters in practical applications, effectively avoiding saddle points. In summary, this paper addresses the limitations of existing methods in handling large condition numbers or rank deficiency by demonstrating the advantages of the ProjGD algorithm in low-rank matrix estimation.

Projected Gradient Descent Algorithm for Low-Rank Matrix Estimation

Convergence Analysis of Projected Gradient Descent for Schatten-p Nonconvex Matrix Recovery

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

Accelerating Ill-Conditioned Low-Rank Matrix Estimation via Scaled Gradient Descent

Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees

Fast Gradient Method for Low-Rank Matrix Estimation

Provable accelerated gradient method for nonconvex low rank optimization

Low-rank matrix recovery with non-quadratic loss: projected gradient method and regularity projection oracle

A Preconditioned Riemannian Gradient Descent Algorithm for Low-Rank Matrix Recovery

Accelerated Gradient Method for A Class of Nonconvex Low Rank Problem: Essentially Matching the Optimal Convex Convergence Rate

Exact Linear Convergence Rate Analysis for Low-Rank Symmetric Matrix Completion via Gradient Descent

Scalable Algorithms for Locally Low-Rank Matrix Modeling

Beyond Procrustes: Balancing-Free Gradient Descent for Asymmetric Low-Rank Matrix Sensing

Gauss-Southwell type descent methods for low-rank matrix optimization

Computationally Efficient and Statistically Optimal Robust Low-rank Matrix and Tensor Estimation

Low-Rank Matrix Estimation from Rank-One Projections by Unlifted Convex Optimization

A Unified Computational and Statistical Framework for Nonconvex Low-rank Matrix Estimation

Global Convergence of a Grassmannian Gradient Descent Algorithm for Subspace Estimation

A Single-Mode Quasi Riemannian Gradient Descent Algorithm for Low-Rank Tensor Recovery

Nonconvex Low-Rank Matrix Recovery with Arbitrary Outliers via Median-Truncated Gradient Descent