Abstract:In this paper, we consider the geometric landscape connection of the widely studied manifold and factorization formulations in low-rank positive semidefinite (PSD) and general matrix optimization. We establish a sandwich relation on the spectrum of Riemannian and Euclidean Hessians at first-order stationary points (FOSPs). As a result of that, we obtain an equivalence on the set of FOSPs, second-order stationary points (SOSPs) and strict saddles between the manifold and the factorization formulations. In addition, we show the sandwich relation can be used to transfer more quantitative geometric properties from one formulation to another. Similarities and differences in the landscape connection under the PSD case and the general case are discussed. To the best of our knowledge, this is the first geometric landscape connection between the manifold and the factorization formulations for handling rank constraints, and it provides a geometric explanation for the similar empirical performance of factorization and manifold approaches in low-rank matrix optimization observed in the literature. In the general low-rank matrix optimization, the landscape connection of two factorization formulations (unregularized and regularized ones) is also provided. By applying these geometric landscape connections, in particular, the sandwich relation, we are able to solve unanswered questions in literature and establish stronger results in the applications on geometric analysis of phase retrieval, well-conditioned low-rank matrix optimization, and the role of regularization in factorization arising from machine learning and signal processing.

Exterior Point Method for Completely Positive Factorization

Towards Optimal Bounds for Implicit Factorization Problem

CIMGS: an Incomplete Orthogonal FactorizationPreconditioner

Local Coordinate Projective Non-Negative Matrix Factorization

Approximate Completely Positive Semidefinite Factorizations and their Ranks

A Symmetric Rank-one Quasi Newton Method for Non-negative Matrix Factorization

Linear Projective Non-Negative Matrix Factorization

Integer Factorization via Continued Fractions and Quadratic Forms

Nonconvex Factorization and Manifold Formulations are Almost Equivalent in Low-rank Matrix Optimization

The Cone of Completely Positive Matrices

A circumcentered-reflection method for finding common fixed points of firmly nonexpansive operators

Factor-Bounded Nonnegative Matrix Factorization

The numerical factorization of polynomials

Convergent Projective Non-Negative Matrix Factorization with Kullback-Leibler Divergence

Efficient Robust Matrix Factorization with Nonconvex Penalties.

An Efficient Nonmonotone Projected Barzilai–Borwein Method for Nonnegative Matrix Factorization with Extrapolation

Efficient Nonnegative Matrix Factorization Via Projected Newton Method.

Nonnegative Tensor Factorization, Completely Positive Tensors and an Hierarchical Elimination Algorithm

The Analytic Computation of Inner-Outer and Spectral Factorization Problems

Convex Algorithms for Nonnegative Matrix Factorization

Greedy Orthogonal Pivoting Algorithm for Non-Negative Matrix Factorization.