Abstract:The Gaussian kernel and its traditional normalizations (e.g., row-stochastic) are popular approaches for assessing similarities between data points. Yet, they can be inaccurate under high-dimensional noise, especially if the noise magnitude varies considerably across the data, e.g., under heteroskedasticity or outliers. In this work, we investigate a more robust alternative -- the doubly stochastic normalization of the Gaussian kernel. We consider a setting where points are sampled from an unknown density on a low-dimensional manifold embedded in high-dimensional space and corrupted by possibly strong, non-identically distributed, sub-Gaussian noise. We establish that the doubly stochastic affinity matrix and its scaling factors concentrate around certain population forms, and provide corresponding finite-sample probabilistic error bounds. We then utilize these results to develop several tools for robust inference under general high-dimensional noise. First, we derive a robust density estimator that reliably infers the underlying sampling density and can substantially outperform the standard kernel density estimator under heteroskedasticity and outliers. Second, we obtain estimators for the pointwise noise magnitudes, the pointwise signal magnitudes, and the pairwise Euclidean distances between clean data points. Lastly, we derive robust graph Laplacian normalizations that accurately approximate various manifold Laplacians, including the Laplace Beltrami operator, improving over traditional normalizations in noisy settings. We exemplify our results in simulations and on real single-cell RNA-sequencing data. For the latter, we show that in contrast to traditional methods, our approach is robust to variability in technical noise levels across cell types.

Unified generalized iterative scaling and its applications

A Unified Bayesian Inference Framework for Generalized Linear Models

Generalized Iterative Scaling for Regularized Optimal Transport with Affine Constraints: Application Examples

Iterative proportional scaling revisited: a modern optimization perspective

On the maximum likelihood estimation in general log-linear models

Levenberg-Marquardt method with Singular Scaling and applications

An Efficient Adaptive Scaling Parameter for the Spectral Conjugate Gradient Method.

Scaling Limits of the Wasserstein information matrix on Gaussian Mixture Models

A Multiscale Perspective on Maximum Marginal Likelihood Estimation

Estimation and prediction for spatial generalized linear mixed models with parametric links via reparameterized importance sampling

Robust Inference of Manifold Density and Geometry by Doubly Stochastic Scaling

Warm Start Marginal Likelihood Optimisation for Iterative Gaussian Processes

A type-II maximum-likelihood approach to Gaussian scale mixture-based sparse regression Kriging

Uncertainty quantification in high-dimensional linear models incorporating graphical structures with applications to gene set analysis

Probabilistic Classifiers with a Generalized Gaussian Scale Mixture Prior

An Information Geometry Interpretation for Approximate Message Passing

Scalable Statistical Inference in Non-parametric Least Squares

Conjugate gradient methods for high-dimensional GLMMs

Manifold Optimization Assisted Gaussian Variational Approximation

A Generalized Alternating Direction Implicit Method for Consensus Optimization: Application to Distributed Sparse Logistic Regression

Distribution-Free Location-Scale Regression