Abstract:This paper considers general rank-constrained optimization problems that minimize a general objective function ${f}( {X})$ over the set of rectangular ${n}times {m}$ matrices that have rank at most r. To tackle the rank constraint and also to reduce the computational burden, we factorize $ {X}$ into $ {U} {V} ^{mathrm {T}}$ where $ {U}$ and $ {V}$ are ${n}times {r}$ and ${m}times {r}$ matrices, respectively, and then optimize over the small matrices $ {U}$ and $ {V}$ . We characterize the global optimization geometry of the nonconvex factored problem and show that the corresponding objective function satisfies the robust strict saddle property as long as the original objective function f satisfies restricted strong convexity and smoothness properties, ensuring global convergence of many local search algorithms (such as noisy gradient descent) in polynomial time for solving the factored problem. We also provide a comprehensive analysis for the optimization geometry of a matrix factorization problem where we aim to find ${n}times {r}$ and ${m}times {r}$ matrices $ {U}$ and $ {V}$ such that $ {U} {V} ^{mathrm {T}}$ approximates a given matrix $ {X}^star $ . Aside from the robust strict saddle property, we show that the objective function of the matrix factorization problem has no spurious local minima and obeys the strict saddle property not only for the exact-parameterization case where $mathrm {rank}( {X}^star) = {r}$ , but also for the over-parameterization case where $mathrm {rank}( {X}^star) < {r}$ and the under-parameterization case where $mathrm {rank}( {X}^star) > {r}$ . These geometric properties imply that a number of iterative optimization algorithms (such as gradient descent) converge to a global solution with random initialization.

Error bounds for rank constrained optimization problems and applications.

Error Bounds for Approximate Solutions of Abstract Inequality Systems and Infinite Systems of Inequalities on Banach Spaces

Porosity of perturbed optimization problems in Banach spaces

Error bound and exact penalty method for optimization problems with nonnegative orthogonal constraint

Quantitative Analysis for Perturbed Abstract Inequality Systems in Banach Spaces

The Equivalence of Three Types of Error Bounds for Weakly and Approximately Convex Functions

Error Bounds for Some Convex Functions and Distance Composite Functions

Error Bounds for Abstract Linear Inequality Systems

Convergence rate of inexact proximal point algorithms for operator with holder metric subregularity

Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

On the Partial Convexification for Low-Rank Spectral Optimization: Rank Bounds and Algorithms

On Error Bounds of Inequalities in Asplund Spaces

General Low-rank Matrix Optimization: Geometric Analysis and Sharper Bounds

The Global Optimization Geometry of Low-Rank Matrix Optimization

Extreme Point Pursuit -- Part II: Further Error Bound Analysis and Applications

Tight Error Bounds for the Sign-Constrained Stiefel Manifold

Perturbation Analysis of Error Bounds for Convex Functions on Banach Spaces

Primal characterizations of stability of error bounds for semi-infinite convex constraint systems in Banach spaces

DISCREPANCY BOUNDS FOR THE DISTRIBUTION OF RANKIN–SELBERG L -FUNCTIONS

Error bounds, PL condition, and quadratic growth for weakly convex functions, and linear convergences of proximal point methods

Upper and Lower Bounds on Quantities of Interest for Contact Problems