Abstract:This paper considers general rank-constrained optimization problems that minimize a general objective function ${f}( {X})$ over the set of rectangular ${n}times {m}$ matrices that have rank at most r. To tackle the rank constraint and also to reduce the computational burden, we factorize $ {X}$ into $ {U} {V} ^{mathrm {T}}$ where $ {U}$ and $ {V}$ are ${n}times {r}$ and ${m}times {r}$ matrices, respectively, and then optimize over the small matrices $ {U}$ and $ {V}$ . We characterize the global optimization geometry of the nonconvex factored problem and show that the corresponding objective function satisfies the robust strict saddle property as long as the original objective function f satisfies restricted strong convexity and smoothness properties, ensuring global convergence of many local search algorithms (such as noisy gradient descent) in polynomial time for solving the factored problem. We also provide a comprehensive analysis for the optimization geometry of a matrix factorization problem where we aim to find ${n}times {r}$ and ${m}times {r}$ matrices $ {U}$ and $ {V}$ such that $ {U} {V} ^{mathrm {T}}$ approximates a given matrix $ {X}^star $ . Aside from the robust strict saddle property, we show that the objective function of the matrix factorization problem has no spurious local minima and obeys the strict saddle property not only for the exact-parameterization case where $mathrm {rank}( {X}^star) = {r}$ , but also for the over-parameterization case where $mathrm {rank}( {X}^star) < {r}$ and the under-parameterization case where $mathrm {rank}( {X}^star) > {r}$ . These geometric properties imply that a number of iterative optimization algorithms (such as gradient descent) converge to a global solution with random initialization.

On the Generalized Low Rank Approximation of the Correlation Matrices Arising in the Asset Portfolio

Optimal low-rank approximation to a correlation matrix

The Efficient Frontier for Weakly Correlated Assets

Kendall Correlation Coefficients for Portfolio Optimization

Portfolio Optimization under Correlation Constraint

Matrix Completion via Non-Convex Relaxation and Adaptive Correlation Learning

The Global Optimization Geometry of Low-Rank Matrix Optimization

Generalized Low-rank plus Sparse Tensor Estimation by Fast Riemannian Optimization

A Dynamic Structure for High Dimensional Covariance Matrices and its Application in Portfolio Allocation

Generalized Low-Rank Matrix Completion Model with Overlapping Group Error Representation

Robust Regularized Low-Rank Matrix Models for Regression and Classification

Low Rank Approximation of Binary Matrices: Column Subset Selection and Generalizations

GoDec+: Fast and Robust Low-Rank Matrix Decomposition Based on Maximum Correntropy.

General Constrained Matrix Optimization

Multivariate Spatiotemporal Models with Low Rank Coefficient Matrix

Feasible Implied Correlation Matrices from Factor Structures

Approximate Factor Models with Strongly Correlated Idiosyncratic Errors

Approximation Conjugate Gradient Method for Low-Rank Matrix Recovery

Cross-Domain Object Representation Via Robust Low-Rank Correlation Analysis.

Large Portfolio Allocation Based on High-Dimensional Regression and Kendall's Tau

An efficient multivariate volatility model for many assets