Abstract:This paper considers general rank-constrained optimization problems that minimize a general objective function ${f}( {X})$ over the set of rectangular ${n}times {m}$ matrices that have rank at most r. To tackle the rank constraint and also to reduce the computational burden, we factorize $ {X}$ into $ {U} {V} ^{mathrm {T}}$ where $ {U}$ and $ {V}$ are ${n}times {r}$ and ${m}times {r}$ matrices, respectively, and then optimize over the small matrices $ {U}$ and $ {V}$ . We characterize the global optimization geometry of the nonconvex factored problem and show that the corresponding objective function satisfies the robust strict saddle property as long as the original objective function f satisfies restricted strong convexity and smoothness properties, ensuring global convergence of many local search algorithms (such as noisy gradient descent) in polynomial time for solving the factored problem. We also provide a comprehensive analysis for the optimization geometry of a matrix factorization problem where we aim to find ${n}times {r}$ and ${m}times {r}$ matrices $ {U}$ and $ {V}$ such that $ {U} {V} ^{mathrm {T}}$ approximates a given matrix $ {X}^star $ . Aside from the robust strict saddle property, we show that the objective function of the matrix factorization problem has no spurious local minima and obeys the strict saddle property not only for the exact-parameterization case where $mathrm {rank}( {X}^star) = {r}$ , but also for the over-parameterization case where $mathrm {rank}( {X}^star) < {r}$ and the under-parameterization case where $mathrm {rank}( {X}^star) > {r}$ . These geometric properties imply that a number of iterative optimization algorithms (such as gradient descent) converge to a global solution with random initialization.

Formulas for calculating the extremal ranks and inertias of a matrix-valued function subject to matrix equation restrictions

Equalities and inequalities for Hermitian solutions and Hermitian definite solutions of the two matrix equations $AX = B$ and $AXA^* = B$

Upper and lower bounds for the rank of the matrix-valued function when X has a fixed rank

The Minimal Rank of Matrix Expressions with Respect to Hermitian Matrix-Revised

Extremal Ranks of Matrix Expression of A - BXC with Respect to Hermitian Matrix.

Extremal Ranks for Block Hermitian and Skew-Hermitian Matrices

Preconditioned Low-Rank Riemannian Optimization for Symmetric Positive Definite Linear Matrix Equations

Minimal Ranks of the Product of Generalized Inverses of Matrices

The Global Optimization Geometry of Low-Rank Matrix Optimization

On the Maximal Solution of the Matrix Equation X+ATX-1A=I

Matrix Nearness Problems with Off-Block-Diagonal Rank Constraints

On the convergence of orthogonalization-free conjugate gradient method for extreme eigenvalues of Hermitian matrices: a Riemannian optimization interpretation

General Low-rank Matrix Optimization: Geometric Analysis and Sharper Bounds

Matrix Function Optimization Problems under Orthonormal Constraint

Exploring Algorithmic Limits of Matrix Rank Minimization under Affine Constraints

Positive Definite Solutions of the Nonlinear Matrix Equation $X+A^{\mathrm{H}}\bar{X}^{-1}A=I$

The Matrix Equations with Constraint

Maximum Likelihood for Matrices with Rank Constraints

Generalized Inverses and a Block-Rank Equation.

A Riemannian optimization method on the indefinite Stiefel manifold

An increasing rank Riemannian method for generalized Lyapunov equations