Abstract:In statistical inference, we commonly assume that samples are independent and identically distributed from a probability distribution included in a pre-specified statistical model. However, such an assumption is often violated in practice. Even an unexpected extreme sample called an {\it outlier} can significantly impact classical estimators. Robust statistics studies how to construct reliable statistical methods that efficiently work even when the ideal assumption is violated. Recently, some works revealed that robust estimators such as Tukey's median are well approximated by the generative adversarial net (GAN), a popular learning method for complex generative models using neural networks. GAN is regarded as a learning method using integral probability metrics (IPM), which is a discrepancy measure for probability distributions. In most theoretical analyses of Tukey's median and its GAN-based approximation, however, the Gaussian or elliptical distribution is assumed as the statistical model. In this paper, we explore the application of GAN-like estimators to a general class of statistical models. As the statistical model, we consider the kernel exponential family that includes both finite and infinite-dimensional models. To construct a robust estimator, we propose the smoothed total variation (STV) distance as a class of IPMs. Then, we theoretically investigate the robustness properties of the STV-based estimators. Our analysis reveals that the STV-based estimator is robust against the distribution contamination for the kernel exponential family. Furthermore, we analyze the prediction accuracy of a Monte Carlo approximation method, which circumvents the computational difficulty of the normalization constant.

Diffeomorphic Measure Matching with Kernels for Generative Modeling

Probabilistic Matching of Real and Generated Data Statistics in Generative Adversarial Networks

Probabilistic Distance Measures in Reproducing Kernel Hibert Space

Statistical Inference for Generative Models with Maximum Mean Discrepancy

Generative and Latent Mean Map Kernels

Kernel Mean Embedding of Probability Measures and its Applications to Functional Data Analysis

Generator Matching: Generative modeling with arbitrary Markov processes

Optimal Transport for Kernel Gaussian Mixture Models

Measure transport with kernel mean embeddings

An appointment with Reproducing Kernel Hilbert Space generated by Generalized Gaussian RBF as $L^2-$measure

Consistent estimation of generative model representations in the data kernel perspective space

Efficient Learning of Generative Models via Finite-Difference Score Matching

Kernel distance measures for time series, random fields and other structured data

Efficient Training of Neural Stochastic Differential Equations by Matching Finite Dimensional Distributions

From Sample Similarity to Ensemble Similarity: Probabilistic Distance Measures in Reproducing Kernel Hilbert Space.

A Bias-Variance-Covariance Decomposition of Kernel Scores for Generative Models

Transient anisotropic kernel for probabilistic learning on manifolds

Dimension Amnesic Pyramid Match Kernel

Robust Estimation for Kernel Exponential Families with Smoothed Total Variation Distances

Analysis of Kernel Mirror Prox for Measure Optimization

Kernel Ordinary Differential Equations