Abstract:Matrix denoising is central to signal processing and machine learning. Its analysis when the matrix to infer has a factorised structure with a rank growing proportionally to its dimension remains a challenge, except when it is rotationally invariant. In this case the information theoretic limits and a Bayes-optimal denoising algorithm, called rotational invariant estimator [1,2], are known. Beyond this setting few results can be found. The reason is that the model is not a usual spin system because of the growing rank dimension, nor a matrix model due to the lack of rotation symmetry, but rather a hybrid between the two. In this paper we make progress towards the understanding of Bayesian matrix denoising when the hidden signal is a factored matrix $XX^\intercal$ that is not rotationally invariant. Monte Carlo simulations suggest the existence of a denoising-factorisation transition separating a phase where denoising using the rotational invariant estimator remains Bayes-optimal due to universality properties of the same nature as in random matrix theory, from one where universality breaks down and better denoising is possible by exploiting the signal's prior and factorised structure, though algorithmically hard. We also argue that it is only beyond the transition that factorisation, i.e., estimating $X$ itself, becomes possible up to sign and permutation ambiguities. On the theoretical side, we combine mean-field techniques in an interpretable multiscale fashion in order to access the minimum mean-square error and mutual information. Interestingly, our alternative method yields equations which can be reproduced using the replica approach of [3]. Using numerical insights, we then delimit the portion of the phase diagram where this mean-field theory is reliable, and correct it using universality when it is not. Our ansatz matches well the numerics when accounting for finite size effects.

Rectangular Rotational Invariant Estimator for General Additive Noise Matrices

Rectangular Rotational Invariant Estimator for High-Rank Matrix Estimation

Low Rank Matrix Recovery with Adversarial Sparse Noise

Bayesian Extensive-Rank Matrix Factorization with Rotational Invariant Priors

Asymptotic Estimates for Spectral Estimators of Rotationally Invariant Matrices

Matrix Inference in Growing Rank Regimes

On the phase diagram of extensive-rank symmetric matrix denoising beyond rotational invariance

Optimality of Approximate Message Passing Algorithms for Spiked Matrix Models with Rotationally Invariant Noise

Adapting to Unknown Noise Distribution in Matrix Denoising

Matrices with Gaussian Noise: Optimal Estimates for Singular Subspace Perturbation

Large information plus noise random matrix models and consistent subspace estimation in large sensor networks

Alternating minimization for generalized rank one matrix sensing: Sharp predictions from a random initialization

Consistency of invariance-based randomization tests

Noisy matrix decomposition via convex relaxation: Optimal rates in high dimensions

Matrix Denoising: Bayes-Optimal Estimators Via Low-Degree Polynomials

The random matrix regime of Maronna's M-estimator for observations corrupted by elliptical noises

Fundamental limits of Non-Linear Low-Rank Matrix Estimation

Matrix quadratic risk of orthogonally invariant estimators for a normal mean matrix

Robust Model Order Selection in Large Dimensional Elliptically Symmetric Noise

Randomized low-rank approximations beyond Gaussian random matrices