Abstract:This paper considers general rank-constrained optimization problems that minimize a general objective function ${f}( {X})$ over the set of rectangular ${n}times {m}$ matrices that have rank at most r. To tackle the rank constraint and also to reduce the computational burden, we factorize $ {X}$ into $ {U} {V} ^{mathrm {T}}$ where $ {U}$ and $ {V}$ are ${n}times {r}$ and ${m}times {r}$ matrices, respectively, and then optimize over the small matrices $ {U}$ and $ {V}$ . We characterize the global optimization geometry of the nonconvex factored problem and show that the corresponding objective function satisfies the robust strict saddle property as long as the original objective function f satisfies restricted strong convexity and smoothness properties, ensuring global convergence of many local search algorithms (such as noisy gradient descent) in polynomial time for solving the factored problem. We also provide a comprehensive analysis for the optimization geometry of a matrix factorization problem where we aim to find ${n}times {r}$ and ${m}times {r}$ matrices $ {U}$ and $ {V}$ such that $ {U} {V} ^{mathrm {T}}$ approximates a given matrix $ {X}^star $ . Aside from the robust strict saddle property, we show that the objective function of the matrix factorization problem has no spurious local minima and obeys the strict saddle property not only for the exact-parameterization case where $mathrm {rank}( {X}^star) = {r}$ , but also for the over-parameterization case where $mathrm {rank}( {X}^star) < {r}$ and the under-parameterization case where $mathrm {rank}( {X}^star) > {r}$ . These geometric properties imply that a number of iterative optimization algorithms (such as gradient descent) converge to a global solution with random initialization.

A Note on a Lower Bound on the Minimum Rank of a Positive Semidefinite Hankel Matrix Rank Minimization Problem

The Bounds of Restricted Isometry Constants for Low Rank Matrices Recovery

On the Schatten P-Quasi-norm Minimization for Low-Rank Matrix Recovery

General Low-rank Matrix Optimization: Geometric Analysis and Sharper Bounds

An approximation theory of matrix rank minimization and its application to quadratic equations

Sums of squares, Hankel index and almost real rank

A Note on the Rank of the Optimal Matrix in Symmetric Toeplitz Matrix Completion Problem

On the Partial Convexification for Low-Rank Spectral Optimization: Rank Bounds and Algorithms

Upper and lower bounds for the rank of the matrix-valued function when X has a fixed rank

Quasi-Hankel low-rank matrix completion: a convex relaxation

Minimizing low-rank models of high-order tensors: Hardness, span, tight relaxation, and applications

The Global Optimization Geometry of Low-Rank Matrix Optimization

A Lower Bound For Minimum Positive Semidefinite Rank By Constructing An Os-Vertex Set For A Given Graph

The lower bounds for the rank of matrices and some sufficient conditions for nonsingular matrices

On the geometric interpretation of the nonnegative rank

Matrix Nearness Problems with Off-Block-Diagonal Rank Constraints

Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization

Large-Scale Convex Minimization with a Low-Rank Constraint

Generalized Singular Value Thresholding Operator to Affine Matrix Rank Minimization Problem

Lower bounds on nonnegative rank via nonnegative nuclear norms

Effective Lower Bounds on the Matrix Rank and Their Applications