Abstract:The Gaussian kernel and its traditional normalizations (e.g., row-stochastic) are popular approaches for assessing similarities between data points. Yet, they can be inaccurate under high-dimensional noise, especially if the noise magnitude varies considerably across the data, e.g., under heteroskedasticity or outliers. In this work, we investigate a more robust alternative -- the doubly stochastic normalization of the Gaussian kernel. We consider a setting where points are sampled from an unknown density on a low-dimensional manifold embedded in high-dimensional space and corrupted by possibly strong, non-identically distributed, sub-Gaussian noise. We establish that the doubly stochastic affinity matrix and its scaling factors concentrate around certain population forms, and provide corresponding finite-sample probabilistic error bounds. We then utilize these results to develop several tools for robust inference under general high-dimensional noise. First, we derive a robust density estimator that reliably infers the underlying sampling density and can substantially outperform the standard kernel density estimator under heteroskedasticity and outliers. Second, we obtain estimators for the pointwise noise magnitudes, the pointwise signal magnitudes, and the pairwise Euclidean distances between clean data points. Lastly, we derive robust graph Laplacian normalizations that accurately approximate various manifold Laplacians, including the Laplace Beltrami operator, improving over traditional normalizations in noisy settings. We exemplify our results in simulations and on real single-cell RNA-sequencing data. For the latter, we show that in contrast to traditional methods, our approach is robust to variability in technical noise levels across cell types.

Manifold fitting algorithm of noisy manifold data based on variable-scale spectral graph

Manifold Fitting under Unbounded Noise

Dataset Denoising Based on Manifold Assumption

Noise-resistant Graph Neural Networks with Manifold Consistency and Label Consistency

Fitting a manifold to data in the presence of large noise

Local Smoothing For Manifold Learning

Robust Inference of Manifold Density and Geometry by Doubly Stochastic Scaling

Manifold Fitting with CycleGAN.

Diffusion-based Semi-supervised Spectral Algorithm for Regression on Manifolds

Adaptive graph regularized nonnegative matrix factorization for data representation

Graph-Based Manifold Frequency Analysis for Denoising

Fusion of Manifold Learning and Spectral Clustering Algorithmwith Applications to Fault Diagnosis

Semi-Supervised Learning with Manifold Fitted Graphs.

Learning Manifold Structures With Subspace Segmentations

Manifold learning with local geometry preserving and global affine transformation

Manifold-based learning and synthesis.

Skeleton Regression: A Graph-Based Approach to Estimation with Manifold Structure

Bi-stochastically normalized graph Laplacian: convergence to manifold Laplacian and robustness to outlier noise

LOCAL LINEAR SMOOTHING FOR NONLINEAR MANIFOLD LEARNING

Investigating Manifold Learning Algorithms Based on Magnification Factors and Principal Spread Directions

Low rank approximation with sparse integration of multiple manifolds for data representation