Abstract:The Gaussian kernel and its traditional normalizations (e.g., row-stochastic) are popular approaches for assessing similarities between data points. Yet, they can be inaccurate under high-dimensional noise, especially if the noise magnitude varies considerably across the data, e.g., under heteroskedasticity or outliers. In this work, we investigate a more robust alternative -- the doubly stochastic normalization of the Gaussian kernel. We consider a setting where points are sampled from an unknown density on a low-dimensional manifold embedded in high-dimensional space and corrupted by possibly strong, non-identically distributed, sub-Gaussian noise. We establish that the doubly stochastic affinity matrix and its scaling factors concentrate around certain population forms, and provide corresponding finite-sample probabilistic error bounds. We then utilize these results to develop several tools for robust inference under general high-dimensional noise. First, we derive a robust density estimator that reliably infers the underlying sampling density and can substantially outperform the standard kernel density estimator under heteroskedasticity and outliers. Second, we obtain estimators for the pointwise noise magnitudes, the pointwise signal magnitudes, and the pairwise Euclidean distances between clean data points. Lastly, we derive robust graph Laplacian normalizations that accurately approximate various manifold Laplacians, including the Laplace Beltrami operator, improving over traditional normalizations in noisy settings. We exemplify our results in simulations and on real single-cell RNA-sequencing data. For the latter, we show that in contrast to traditional methods, our approach is robust to variability in technical noise levels across cell types.

Manifold-based transformation of the probability distribution for convergence in optimization

Manifold Optimization for Gaussian Mixture Models

Optimization on Multiple Manifolds

Manifold Learning for Accelerating Coarse-Grained Optimization

Global Optimization with Orthogonality Constraints Via Stochastic Diffusion on Manifold

Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions

Manifold Optimization Assisted Gaussian Variational Approximation

Extrinsic Bayesian Optimizations on Manifolds

Learning the Solution Manifold in Optimization and Its Application in Motion Planning

Improving Diffusion Models for Inverse Problems using Manifold Constraints

Finding Transition Pathways on Manifolds

Communication Efficient Parallel Algorithms for Optimization on Manifolds

Entropy-based Closure for Probabilistic Learning on Manifolds

Robust Inference of Manifold Density and Geometry by Doubly Stochastic Scaling

Distributed Optimization and Sampling with Constraints from Machine Learning

Manifold Fitting under Unbounded Noise

Convergences for Minimax Optimization Problems over Infinite-Dimensional Spaces Towards Stability in Adversarial Training

Manifold-Constrained Geometric Optimization via Local Parameterizations

A Generalization Result for Convergence in Learning-to-Optimize

Convergence of denoising diffusion models under the manifold hypothesis

Computationally Efficient Learning of Statistical Manifolds