Abstract:Detecting and recovering a low-rank signal in a noisy data matrix is a fundamental task in data analysis. Typically, this task is addressed by inspecting and manipulating the spectrum of the observed data, e.g., thresholding the singular values of the data matrix at a certain critical level. This approach is well-established in the case of homoskedastic noise, where the noise variance is identical across the entries. However, in numerous applications, the noise can be heteroskedastic, where the noise characteristics may vary considerably across the rows and columns of the data. In this scenario, the spectral behavior of the noise can differ significantly from the homoskedastic case, posing various challenges for signal detection and recovery. To address these challenges, we develop an adaptive normalization procedure that equalizes the average noise variance across the rows and columns of a given data matrix. Our proposed procedure is data-driven and fully automatic, supporting a broad range of noise distributions, variance patterns, and signal structures. Our approach relies on recent results in random matrix theory, which describe the resolvent of the noise via the so-called Dyson equation. By leveraging this relation, we can accurately infer the noise level in each row and each column directly from the resolvent of the data. We establish that in many cases, our normalization enforces the standard spectral behavior of homoskedastic noise -- the Marchenko-Pastur (MP) law, allowing for simple and reliable detection of signal components. Furthermore, we demonstrate that our approach can substantially improve signal recovery in heteroskedastic settings by manipulating the spectrum after normalization. Lastly, we apply our method to single-cell RNA sequencing and spatial transcriptomics data, showcasing accurate fits to the MP law after normalization.

On Single-Channel Noise Reduction with Rank-Deficient Noise Correlation Matrix

Low Rank Matrix Recovery with Adversarial Sparse Noise

On the Importance of the Pearson Correlation Coefficient in Noise Reduction

CONVERGENCE AND STABILITY OF ITERATIVELY REWEIGHTED LEAST SQUARES FOR LOW-RANK MATRIXRECOVERY

Optimal Phase-Space Projection For Noise Reduction

Research on a Noise Reduction Method Based on Multi-Resolution Singular Value Decomposition

Low-Frequency Noise Suppression of Seismic Signals Using a Novel Framework Composed of Rank Residual Constraint and Enhanced Block Matching

Seismic Noise Attenuation Using an Online Subspace Tracking Algorithm

An Improved Nonnegative Matrix Factorization Algorithm Combined with K-Means for Audio Noise Reduction

Joint NN-Supported Multichannel Reduction of Acoustic Echo, Reverberation and Noise

Matrix Denoising with Doubly Heteroscedastic Noise: Fundamental Limits and Optimal Spectral Methods

High-Fidelity Noise Reduction with Differentiable Signal Processing

An eigenvalue filtering based subspace approach for speech enhancement

Coherence-free Entrywise Estimation of Eigenvectors in Low-rank Signal-plus-noise Matrix Models

Incoherent Noise Suppression of Seismic Data Based on Robust Low-Rank Approximation

The Dyson Equalizer: Adaptive Noise Stabilization for Low-Rank Signal Detection and Recovery

Noise Reduction Using Sparsity Constrained and Regularized Iterative Thresholding Algorithm and Dictionary

Comparative Analysis Of Discriminative Deep Learning-Based Noise Reduction Methods In Low SNR Scenarios

Multichannel Identification and Nonnegative Equalization for Dereverberation and Noise Reduction Based on Convolutive Transfer Function.

Improved Non-Negative Matrix Factorization-Based Noise Reduction of Leakage Acoustic Signals