Abstract:The Gaussian kernel and its traditional normalizations (e.g., row-stochastic) are popular approaches for assessing similarities between data points. Yet, they can be inaccurate under high-dimensional noise, especially if the noise magnitude varies considerably across the data, e.g., under heteroskedasticity or outliers. In this work, we investigate a more robust alternative -- the doubly stochastic normalization of the Gaussian kernel. We consider a setting where points are sampled from an unknown density on a low-dimensional manifold embedded in high-dimensional space and corrupted by possibly strong, non-identically distributed, sub-Gaussian noise. We establish that the doubly stochastic affinity matrix and its scaling factors concentrate around certain population forms, and provide corresponding finite-sample probabilistic error bounds. We then utilize these results to develop several tools for robust inference under general high-dimensional noise. First, we derive a robust density estimator that reliably infers the underlying sampling density and can substantially outperform the standard kernel density estimator under heteroskedasticity and outliers. Second, we obtain estimators for the pointwise noise magnitudes, the pointwise signal magnitudes, and the pairwise Euclidean distances between clean data points. Lastly, we derive robust graph Laplacian normalizations that accurately approximate various manifold Laplacians, including the Laplace Beltrami operator, improving over traditional normalizations in noisy settings. We exemplify our results in simulations and on real single-cell RNA-sequencing data. For the latter, we show that in contrast to traditional methods, our approach is robust to variability in technical noise levels across cell types.

Stability of Sequential Lateration and of Stress Minimization in the Presence of Noise

Stable network inference in high-dimensional graphical model using single-linkage

Robust Inference of Manifold Density and Geometry by Doubly Stochastic Scaling

On the local and global minimizers of the smooth stress function in Euclidean Distance Matrix problems

A Stability Principle for Learning under Non-Stationarity

Evaluating the Stability of Deep Learning Latent Feature Spaces

Scaling Limit, Noise, Stability

Size Should not Matter: Scale-invariant Stress Metrics

Posterior consistency of semi-supervised regression on graphs

Bi-stochastically normalized graph Laplacian: convergence to manifold Laplacian and robustness to outlier noise

Stability under scaling in the local phases of multiplicative functions

Consistency of empirical distributions of sequences of graph statistics in networks with dependent edges

Stability and convergence of in time approximations of hyperbolic elastodynamics via stepwise minimization

The Stability of Persistence Diagrams Under Non-Uniform Scaling

Majorizing Stress Formula Two

The stability and instability of partial realizations

On the role of eigenvalue disparity and coordinate transformations in the reduction of the linear noise approximation

Strong Consistency, Graph Laplacians, and the Stochastic Block Model

Topology-Preserving Scaling in Data Augmentation

Stable quadrangulations and stable spheres

Local sign stability and its implications for spectra of sparse random graphs and stability of ecosystems