Abstract:Matrix factorization is an inference problem that has acquired importance due to its vast range of applications that go from dictionary learning to recommendation systems and machine learning with deep networks. The study of its fundamental statistical limits represents a true challenge, and despite a decade-long history of efforts in the community, there is still no closed formula able to describe its optimal performances in the case where the rank of the matrix scales linearly with its size. In the present paper, we study this extensive rank problem, extending the alternative 'decimation' procedure that we recently introduced, and carry out a thorough study of its performance. Decimation aims at recovering one column/line of the factors at a time, by mapping the problem into a sequence of neural network models of associative memory at a tunable temperature. Though being sub-optimal, decimation has the advantage of being theoretically analyzable. We extend its scope and analysis to two families of matrices. For a large class of compactly supported priors, we show that the replica symmetric free entropy of the neural network models takes a universal form in the low temperature limit. For sparse Ising prior, we show that the storage capacity of the neural network models diverges as sparsity in the patterns increases, and we introduce a simple algorithm based on a ground state search that implements decimation and performs matrix factorization, with no need of an informative initialization.

Invariance properties of thematic factorizations of matrix functions

Monotone thematic factorizations of matrix functions

Modular Invariance and Anomaly Cancellation Formulas

Regular approximate factorization of a class of matrix-function with an unstable set of partial indices

Vanishing of quasi-invariant generalized functions

Non-denseness of factorable matrix functions

The biholomorphic invariance of essential normality on bounded symmetric domains

Invariant rational functions under rational transformations

Matrix Factorizations via the Inverse Function Theorem

Integration of invariant matrices and application to statistics

Novel Theory for Polynomial and Rational Matrices at Infinity. Part 2. Rational Matrices

Matrix factorization for multivariate time series analysis

An effective criterion for a stable factorisation of strictly nonsingular 2x2 matrix functions. Utilisation of the ExactMPF package

Novel Theory for Polynomial and Rational Matrices at Infinity. Part1. Polynomial Matrices

Factorizations into normal matrices in indefinite inner product spaces

A Bivariate Invariance Principle

Negativity-preserving transforms of tuples of symmetric matrices

The principle of invariance in the Donsker form to the partial sum processes of finite order moving averages

The Decimation Scheme for Symmetric Matrix Factorization

Intrinsic dimensional functional inequalities on model spaces