Abstract:The Gaussian kernel and its traditional normalizations (e.g., row-stochastic) are popular approaches for assessing similarities between data points. Yet, they can be inaccurate under high-dimensional noise, especially if the noise magnitude varies considerably across the data, e.g., under heteroskedasticity or outliers. In this work, we investigate a more robust alternative -- the doubly stochastic normalization of the Gaussian kernel. We consider a setting where points are sampled from an unknown density on a low-dimensional manifold embedded in high-dimensional space and corrupted by possibly strong, non-identically distributed, sub-Gaussian noise. We establish that the doubly stochastic affinity matrix and its scaling factors concentrate around certain population forms, and provide corresponding finite-sample probabilistic error bounds. We then utilize these results to develop several tools for robust inference under general high-dimensional noise. First, we derive a robust density estimator that reliably infers the underlying sampling density and can substantially outperform the standard kernel density estimator under heteroskedasticity and outliers. Second, we obtain estimators for the pointwise noise magnitudes, the pointwise signal magnitudes, and the pairwise Euclidean distances between clean data points. Lastly, we derive robust graph Laplacian normalizations that accurately approximate various manifold Laplacians, including the Laplace Beltrami operator, improving over traditional normalizations in noisy settings. We exemplify our results in simulations and on real single-cell RNA-sequencing data. For the latter, we show that in contrast to traditional methods, our approach is robust to variability in technical noise levels across cell types.

Estimating Shape Distances on Neural Representations with Limited Samples

Generalized Shape Metrics on Neural Representations

Full and Partial Shape Similarity Through Sparse Descriptor Reconstruction

Duality of Bures and Shape Distances with Implications for Comparing Neural Representations

Soft Matching Distance: A metric on neural representations that captures single-neuron tuning

Neural networks learn to magnify areas near decision boundaries

From Images to Probabilistic Anatomical Shapes: A Deep Variational Bottleneck Approach

Shape Analysis with Hyperbolic Wasserstein Distance

Few-Shot Unsupervised Implicit Neural Shape Representation Learning with Spatial Adversaries

Statistical inference on representational geometries

Shape-informed surrogate models based on signed distance function domain encoding

Robust Inference of Manifold Density and Geometry by Doubly Stochastic Scaling

Latent Space Representation for Shape Analysis and Learning

Scalable Stochastic Gradient Riemannian Langevin Dynamics in Non-Diagonal Metrics

Improved Estimation of Concentration Under $\ell_p$-Norm Distance Metrics Using Half Spaces

Combining Statistical Depth and Fermat Distance for Uncertainty Quantification

Neural Estimation Of Entropic Optimal Transport

Probabilistic Directed Distance Fields for Ray-Based Shape Representations

Deep data density estimation through Donsker-Varadhan representation

On the Convergence of Locally Adaptive and Scalable Diffusion-Based Sampling Methods for Deep Bayesian Neural Network Posteriors

A statistical framework for analyzing shape in a time series of random geometric objects