Abstract:This paper considers general rank-constrained optimization problems that minimize a general objective function ${f}( {X})$ over the set of rectangular ${n}times {m}$ matrices that have rank at most r. To tackle the rank constraint and also to reduce the computational burden, we factorize $ {X}$ into $ {U} {V} ^{mathrm {T}}$ where $ {U}$ and $ {V}$ are ${n}times {r}$ and ${m}times {r}$ matrices, respectively, and then optimize over the small matrices $ {U}$ and $ {V}$ . We characterize the global optimization geometry of the nonconvex factored problem and show that the corresponding objective function satisfies the robust strict saddle property as long as the original objective function f satisfies restricted strong convexity and smoothness properties, ensuring global convergence of many local search algorithms (such as noisy gradient descent) in polynomial time for solving the factored problem. We also provide a comprehensive analysis for the optimization geometry of a matrix factorization problem where we aim to find ${n}times {r}$ and ${m}times {r}$ matrices $ {U}$ and $ {V}$ such that $ {U} {V} ^{mathrm {T}}$ approximates a given matrix $ {X}^star $ . Aside from the robust strict saddle property, we show that the objective function of the matrix factorization problem has no spurious local minima and obeys the strict saddle property not only for the exact-parameterization case where $mathrm {rank}( {X}^star) = {r}$ , but also for the over-parameterization case where $mathrm {rank}( {X}^star) < {r}$ and the under-parameterization case where $mathrm {rank}( {X}^star) > {r}$ . These geometric properties imply that a number of iterative optimization algorithms (such as gradient descent) converge to a global solution with random initialization.

Equivalence between the affine matrix rank minimization problem and the unconstrained matrix rank minimization problem

Promoting low rank non-convex fraction function penalty for affine matrix rank minimization

A New Nonconvex Strategy to Affine Matrix Rank Minimization Problem

On the Schatten P-Quasi-norm Minimization for Low-Rank Matrix Recovery

Exact Recovery Low-Rank Matrix Via Transformed Affine Matrix Rank Minimization

Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization

DC algorithm for solving the transformed affine matrix rank minimization

Multistage Convex Relaxation Approach to Rank Regularized Minimization Problems Based on Equivalent Mathematical Program with a Generalized Complementarity Constraint

Generalized Singular Value Thresholding Operator to Affine Matrix Rank Minimization Problem

Exploring Algorithmic Limits of Matrix Rank Minimization under Affine Constraints

An approximation theory of matrix rank minimization and its application to quadratic equations

A continuous-time neurodynamic approach in matrix form for rank minimization

An Efficient Method for Convex Constrained Rank Minimization Problems Based on Dc Programming

On the equivalence between low rank matrix completion and tensor rank

The Rate of Convergence of Augmented Lagrangian Method for Minimax Optimization Problems with Equality Constraints

The Global Optimization Geometry of Low-Rank Matrix Optimization

Large-Scale Affine Matrix Rank Minimization with a Novel Nonconvex Regularizer.

The lower bounds for the rank of matrices and some sufficient conditions for nonsingular matrices

A Variable Metric Algorithm with Broyden Rank One Modifications for Nonlinear Equality Constraints Optimization

Approximation of Rank Function and Its Application to the Nearest Low-Rank Correlation Matrix

Stochastic Variance Reduced Gradient for Affine Rank Minimization Problem