Abstract:We present a unified theory for Mahalanobis-type anomaly detection on Banach spaces, using ideas from Cameron-Martin theory applied to non-Gaussian measures. This approach leads to a basis-free, data-driven notion of anomaly distance through the so-called variance norm of a probability measure, which can be consistently estimated using empirical measures. Our framework generalizes the classical $\mathbb{R}^d$, functional $(L^2[0,1])^d$, and kernelized settings, including the general case of non-injective covariance operator. We prove that the variance norm depends solely on the inner product in a given Hilbert space, and hence that the kernelized Mahalanobis distance can naturally be recovered by working on reproducing kernel Hilbert spaces. Using the variance norm, we introduce the notion of a kernelized nearest-neighbour Mahalanobis distance for semi-supervised anomaly detection. In an empirical study on 12 real-world datasets, we demonstrate that the kernelized nearest-neighbour Mahalanobis distance outperforms the traditional kernelized Mahalanobis distance for multivariate time series anomaly detection, using state-of-the-art time series kernels such as the signature, global alignment, and Volterra reservoir kernels. Moreover, we provide an initial theoretical justification of nearest-neighbour Mahalanobis distances by developing concentration inequalities in the finite-dimensional Gaussian case.

An invariance property of kernel based predictors

Integrated Latent Heterogeneity and Invariance Learning in Kernel Space

Kernelized Heterogeneous Risk Minimization

Covariate shift in nonparametric regression with Markovian design

Invariant Risk Minimization Games

The Optimality of Kernel Classifiers in Sobolev Space

Optimally tackling covariate shift in RKHS-based nonparametric regression

The Exact Sample Complexity Gain from Invariances for Kernel Regression

Towards a Unified Analysis of Kernel-based Methods Under Covariate Shift

On kernel methods for covariates that are rankings

Conformal Inference for Invariant Risk Minimization

Invariant Probabilistic Prediction

Generalization in Kernel Regression Under Realistic Assumptions

Variance Norms for Kernelized Anomaly Detection

A non-asymptotic theory of Kernel Ridge Regression: deterministic equivalents, test error, and GCV estimator

A theory of data variability in Neural Network Bayesian inference

Kernel ridge vs. principal component regression: minimax bounds and adaptability of regularization operators

On Kernel Regression with Data-Dependent Kernels

Invariance of Gaussian RKHSs under Koopman operators of stochastic differential equations with constant matrix coefficients

Learning Adaptive Kernels for Statistical Independence Tests.

Kernel regression, minimax rates and effective dimensionality: beyond the regular case