Bounding Hellinger Distance with Stein's Method

Morgane Austern,Lester Mackey

2024-11-07

Abstract:This work introduces a new, explicit bound on the Hellinger distance between a continuous random variable and a Gaussian with matching mean and variance. As example applications, we derive a quantitative Hellinger central limit theorem and efficient concentration inequalities for sums of dependent random variables.

Probability

What problem does this paper attempt to address?

On Bernstein Type Inequalities for Stochastic Integrals of Multivariate Point Processes

Hanchao Wang,Zhengyan Lin,Zhonggen Su

DOI: https://doi.org/10.1016/j.spa.2018.05.014

IF: 1.43

2019-01-01

Stochastic Processes and their Applications

Abstract:In this paper, we first obtain a Bernstein type of concentration inequality for stochastic integrals of multivariate point processes under some conditions through the Doléans-Dade exponential formula, and then derive a uniform exponential inequality using a generic chaining argument. As a direct consequence, we obtain an upper bound for a sequence of discrete time martingales indexed by a class of functionals. Finally, we apply the uniform exponential bound to nonparametric maximum likelihood estimators and provide a rate of convergence in terms of Hellinger distance, which is an improvement of earlier work of van de Geer (1995).
Bounding the -Distance Between One-Dimensional Continuous and Discrete Distributions via Stein's Method

Gilles Germain,Yvik Swan

DOI: https://doi.org/10.1007/s10959-024-01373-x

2024-11-04

Journal of Theoretical Probability

Abstract:We introduce a new version of Stein's method of comparison of operators specifically tailored to the problem of bounding the (a.k.a. Wasserstein-1) distance between continuous and discrete distributions on the real line. Our approach rests on a new family of weighted discrete derivative operators, which we call bespoke derivatives. We also propose new bounds on the derivatives of the solutions of Stein equations for integrated Pearson random variables; this is a crucial step in Stein's method. We apply our result to several examples, including the central limit theorem, Pólya–Eggenberger urn models, the empirical distribution of the ground state of a many-interacting-worlds harmonic oscillator, the stationary distribution for the number of genes in the Moran model, and the stationary distribution of the Erlang-C system. Whenever our bounds can be compared with bounds from the literature, our constants are sharper.

statistics & probability
Efficient Concentration with Gaussian Approximation

Morgane Austern,Lester Mackey

2024-05-22

Abstract:Concentration inequalities for the sample mean, like those due to Bernstein and Hoeffding, are valid for any sample size but overly conservative, yielding confidence intervals that are unnecessarily wide. The central limit theorem (CLT) provides asymptotic confidence intervals with optimal width, but these are invalid for all sample sizes. To resolve this tension, we develop new computable concentration inequalities with asymptotically optimal size, finite-sample validity, and sub-Gaussian decay. These bounds enable the construction of efficient confidence intervals with correct coverage for any sample size and efficient empirical Berry-Esseen bounds that require no prior knowledge of the population variance. We derive our inequalities by tightly bounding non-uniform Kolmogorov and Wasserstein distances to a Gaussian using zero-bias couplings and Stein's method of exchangeable pairs.

Probability,Statistics Theory
Quantitative and stable limits of high-frequency statistics of Lévy processes: a Stein's method approach

Chiara Amorino,Arturo Jaramillo,Mark Podolskij

2024-04-25

Abstract:We establish inequalities for assessing the distance between the distribution of errors of partially observed high-frequency statistics of multidimensional Lévy processes and that of a mixed Gaussian random variable. Furthermore, we provide a general result guaranteeing stable functional convergence. Our arguments rely on a suitable adaptation of the Stein's method perspective to the context of mixed Gaussian distributions, specifically tailored to the framework of high-frequency statistics.

Probability
One-dimensional Stein's method with bespoke derivatives

Gilles Germain,Yvik Swan

2023-11-03

Abstract:We introduce a version of Stein's method of comparison of operators specifically tailored to the problem of bounding the Wasserstein-1 distance between continuous and discrete distributions on the real line. Our approach rests on a new family of weighted discrete derivative operators, which we call bespoke derivatives. We also propose new bounds on the derivatives of the solutions of Stein equations for Integrated Pearson random variables; this is a crucial step in Stein's method. We apply our result to several examples, including the Central Limit Theorem, Polya-Eggenberger urn models, the empirical distribution of the ground state of a many-interacting-worlds harmonic oscillator, the stationary distribution for the number of genes in the Moran model, and the stationary distribution of the Erlang-C system. Whenever our bounds can be compared with bounds from the literature, our constants are sharper.

Probability
Maximum interpoint distance of high-dimensional random vectors

Johannes Heiny,Carolin Kleemann

2024-02-12

Abstract:A limit theorem for the largest interpoint distance of $p$ independent and identically distributed points in $\mathbb{R}^n$ to the Gumbel distribution is proved, where the number of points $p=p_n$ tends to infinity as the dimension of the points $n\to\infty$. The theorem holds under moment assumptions and corresponding conditions on the growth rate of $p$. We obtain a plethora of ancillary results such as the joint convergence of maximum and minimum interpoint distances. Using the inherent sum structure of interpoint distances, our result is generalized to maxima of dependent random walks with non-decaying correlations and we also derive point process convergence. An application of the maximum interpoint distance to testing the equality of means for high-dimensional random vectors is presented. Moreover, we study the largest off-diagonal entry of a sample covariance matrix. The proofs are based on the Chen-Stein Poisson approximation method and Gaussian approximation to large deviation probabilities.

Probability,Statistics Theory
Stein's density method for multivariate continuous distributions

Guillaume Mijoule,Martin Raič,Gesine Reinert,Yvik Swan

DOI: https://doi.org/10.48550/arXiv.2101.05079

2023-04-26

Abstract:This paper provides a general framework for Stein's density method for multivariate continuous distributions. The approach associates to any probability density function a canonical operator and Stein class, as well as an infinite collection of operators and classes which we call standardizations. These in turn spawn an entire family of Stein identities and characterizations for any continuous distribution on $\mathbb{R}^d$, among which we highlight those based on the score function and the Stein kernel. A feature of these operators is that they do not depend on normalizing constants. A new definition of Stein kernel is introduced and examined; integral formulas are obtained through a connection with mass transport, as well as ready-to-use explicit formulas for elliptical distributions. The flexibility of the kernels is used to compare in Stein discrepancy (and therefore 2-Wasserstein distance) between two normal distributions, Student and normal distributions, as well as two normal-gamma distributions. Upper and lower bounds on the 1-Wasserstein distance between continuous distributions are provided, and computed for a variety of examples: comparison between different normal distributions (improving on existing bounds in some regimes), posterior distributions with different priors in a Bayesian setting (including logistic regression), centred Azzalini--Dalla Valle distributions. Finally the notion of weak Stein equation and weak Stein factors is introduced. Bounds for solutions of the weak Stein equation are obtained for Lipschitz test functions if the distribution admits a Poincaré constant. We use these bounds to compare different copulas on the unit square in 1-Wasserstein distance.

Probability
A minimum Hellinger distance estimator for stochastic differential equations: An application to statistical inference for continuous time interest rate models

Ludovic Giet,Michel Lubrano

DOI: https://doi.org/10.1016/j.csda.2007.10.004

2008-02-01

Abstract:A minimum disparity estimator minimizes a φ-divergence between the marginal density of a parametric model and its non-parametric estimate. This principle is applied to the estimation of stochastic differential equation models, choosing the Hellinger distance as particular φ-divergence. Under an hypothesis of stationarity, the parametric marginal density is provided by solving the Kolmogorov forward equation. A particular emphasis is put on the non-parametric estimation of the sample marginal density which has to take into account sample dependence and kurtosis. A new window size determination is provided. The classical estimator is presented alternatively as a distance minimizer and as a pseudo-likelihood maximizer. The latter presentation opens the way to Bayesian inference. The method is applied to continuous time models of the interest rate. In particular, various models are tested using alternatively tests and their results are discussed.

statistics & probability,computer science, interdisciplinary applications
Total variation bound for Hadwiger's functional using Stein's method

Valentin Garino,Ivan Nourdin

2023-04-13

Abstract:Let $K$ be a convex body in $\mathbb{R}^d$. Let $X_K$ be a $d$-dimensional random vector distributed according to the Hadwiger-Wills density $\mu_K$ associated with $K$, defined as $\mu_K(x)=ce^{-\pi {\rm dist}^2(x,K)}$, $x\in \mathbb{R}^d$. Finally, let the information content $H_K$ be defined as $H_K={\rm dist}^2(X_K,K)$. The goal of this paper is to study the fluctuations of $H_K$ around its expectation as the dimension $d$ go to infinity. Relying on Stein's method and Brascamp-Lieb inequality, we compute an explicit bound for the total variation distance between $H_K$ and its Gaussian counterpart.

Probability
Bounds for distributional approximation in the multivariate delta method by Stein's method

Robert E. Gaunt,Heather Sutcliffe

2023-05-10

Abstract:We obtain bounds to quantify the distributional approximation in the delta method for vector statistics (the sample mean of $n$ independent random vectors) for normal and non-normal limits, measured using smooth test functions. For normal limits, we obtain bounds of the optimal order $n^{-1/2}$ rate of convergence, but for a wide class of non-normal limits, which includes quadratic forms amongst others, we achieve bounds with a faster order $n^{-1}$ convergence rate. We apply our general bounds to derive explicit bounds to quantify distributional approximations of an estimator for Bernoulli variance, several statistics of sample moments, order $n^{-1}$ bounds for the chi-square approximation of a family of rank-based statistics, and we also provide an efficient independent derivation of an order $n^{-1}$ bound for the chi-square approximation of Pearson's statistic. In establishing our general results, we generalise recent results on Stein's method for functions of multivariate normal random vectors to vector-valued functions and sums of independent random vectors whose components may be dependent. These bounds are widely applicable and are of independent interest.

Statistics Theory
Multidimensional Stein's method for Gamma approximation

Ciprian A Tudor,Jérémy Zurcher

2023-05-09

Abstract:Let F ($\nu$) be the centered Gamma law with parameter $\nu$ > 0 and let us denote by P Y the probability distribution of a random vector Y. We develop a multidimensional variant of the Stein's method for Gamma approximation that allows to obtain bounds for the second Wasserstein distance between the probability distribution of an arbitrary random vector (X, Y) in R x R n and the probability distribution F ($\nu$) $\otimes$ P Y. In the case of random vectors with components in Wiener chaos, these bounds lead to some interesting criteria for the joint convergence of a sequence ((X n , Y n), n $\ge$ 1) to F ($\nu$) $\otimes$ P Y , by assuming that (X n , n $\ge$ 1) converges in law, as n $\rightarrow$ $\infty$, to F ($\nu$) and (Y n , n $\ge$ 1) converges in law, as n $\rightarrow$ $\infty$, to an arbitrary random vector Y. We illustrate our criteria by two concrete examples.

Probability
Stein's method, smoothing and functional approximation

A. D. Barbour,Nathan Ross,Guangqu Zheng

DOI: https://doi.org/10.1214/24-EJP1081

2023-09-05

Abstract:Stein's method for Gaussian process approximation can be used to bound the differences between the expectations of smooth functionals $h$ of a càdlàg random process $X$ of interest and the expectations of the same functionals of a well understood target random process $Z$ with continuous paths. Unfortunately, the class of smooth functionals for which this is easily possible is very restricted. Here, we prove an infinite dimensional Gaussian smoothing inequality, which enables the class of functionals to be greatly expanded -- examples are Lipschitz functionals with respect to the uniform metric, and indicators of arbitrary events -- in exchange for a loss of precision in the bounds. Our inequalities are expressed in terms of the smooth test function bound, an expectation of a functional of $X$ that is closely related to classical tightness criteria, a similar expectation for $Z$, and, for the indicator of a set $K$, the probability $\mathbb{P}(Z \in K^\theta \setminus K^{-\theta})$ that the target process is close to the boundary of $K$.

Probability,Statistics Theory
Error Bounds on a Mixed Entropy Inequality

James Melbourne,Saurav Talukdar,Shreyas Bhaban,Murti V. Salapaka

DOI: https://doi.org/10.1109/isit.2018.8437601

2018-06-01

Abstract:Motivated by the entropy computations relevant to the evaluation of decrease in entropy in bit reset operations, the authors investigate the deficit in an entropic inequality involving two independent random variables, one continuous and the other discrete. In the case where the continuous random variable is Gaussian, we derive strong quantitative bounds on the deficit in the inequality. More explicitly it is shown that the decay of the deficit is sub-Gaussian with respect to the reciprocal of the standard deviation of the Gaussian variable. What is more, up to rational terms these results are shown to be sharp.
Stein's Method for Spatial Random Graphs

Dominic Schuhmacher,Leoni Carla Wirth

2024-11-05

Abstract:In this article, we derive Stein's method for approximating a spatial random graph by a generalised random geometric graph, which has vertices given by a finite Gibbs point process and edges based on a general connection function. Our main theorems provide explicit upper bounds for integral probability metrics and, at improved rates, a recently introduced Wasserstein metric for random graph distributions. The bounds are in terms of a vertex error term based on the Papangelou kernels of the vertex point processes and two edge error terms based on conditional edge probabilities. In addition to providing new tools for spatial random graphs along the way, such as a graph-based Georgii--Nguyen--Zessin formula, we also give applications of our bounds to the percolation graph of large balls in a Boolean model and to discretising a generalised random geometric graph.

Probability
Gaussian random field approximation via Stein's method with applications to wide random neural networks

Krishnakumar Balasubramanian,Larry Goldstein,Nathan Ross,Adil Salim

2024-05-01

Abstract:We derive upper bounds on the Wasserstein distance ($W_1$), with respect to $\sup$-norm, between any continuous $\mathbb{R}^d$ valued random field indexed by the $n$-sphere and the Gaussian, based on Stein's method. We develop a novel Gaussian smoothing technique that allows us to transfer a bound in a smoother metric to the $W_1$ distance. The smoothing is based on covariance functions constructed using powers of Laplacian operators, designed so that the associated Gaussian process has a tractable Cameron-Martin or Reproducing Kernel Hilbert Space. This feature enables us to move beyond one dimensional interval-based index sets that were previously considered in the literature. Specializing our general result, we obtain the first bounds on the Gaussian random field approximation of wide random neural networks of any depth and Lipschitz activation functions at the random field level. Our bounds are explicitly expressed in terms of the widths of the network and moments of the random weights. We also obtain tighter bounds when the activation function has three bounded derivatives.

Probability,Machine Learning,Statistics Theory
Sharp supremum and Hölder bounds for stochastic integrals indexed by a parameter

Sonja Cox,Joris van Winden

2024-09-21

Abstract:We provide sharp bounds for the supremum of countably many stochastic convolutions taking values in a 2-smooth Banach space. As a consequence, we obtain sharp bounds on the modulus of continuity of a family of stochastic integrals indexed by parameter $x\in M$, where $M$ is a metric space with finite doubling dimension. In particular, we obtain a theory of stochastic integration in Hölder spaces on arbitrary bounded subsets of $\mathbb{R}^d$. This is done by relating the (generalized) Hölder-seminorm associated with a modulus of continuity to a supremum over countably many variables, using a Kolmogorov-type chaining argument. We provide two applications of our results: first, we show long-term bounds for Ornstein-Uhlenbeck processes, and second, we derive novel results regarding the modulus of continuity of the parabolic Anderson model.

Probability,Functional Analysis
New bounds for normal approximation on product spaces with applications to monochromatic edges, random sums and an infinite de Jong CLT

Christian Döbler

2024-03-22

Abstract:We extend the Malliavin theory for $L^2$-functionals on product probability spaces that has recently been developed by Decreusefond and Halconruy (2019) and by Duerinckx (2021), by characterizing the domains and investigating the actions of the three Malliavin operators in terms of the infinite Hoeffding decomposition in $L^2$, which we identify as the natural analogue of the famous Wiener-Itô chaos decomposition on Gaussian and Poisson spaces. We further combine this theory with Stein's method for normal approximation in order to provide three different types of abstract Berry-Esseen and Wasserstein bounds: a) Malliavin-Stein bounds involving the Malliavin gradient $D$ and the pseudo-inverse of the Ornstein-Uhlenbeck generator $L$, b) bounds featuring the carré-du-champ operator $\Gamma$ and c) bounds making use of a Clark-Ocone type integration-by-parts formula. To demonstrate the flexibility of these abstract bounds, we derive quantitative central limit theorems for the number of monochromatic edges in a uniform random coloring of a graph sequence as well as for random sums and prove an infinite version of the quantitative de Jong CLT that has recently been proved by G. Peccati and the author (2017) and by the author (2023). As a further theoretical application, we deduce new abstract Berry-Esseen and Wasserstein bounds for functionals of a general independent Rademacher sequence.

Probability
Log-Harnack Inequality for Stochastic Differential Equations in Hilbert Spaces and Its Consequences

Micahel Röckner,Feng‐Yu Wang

DOI: https://doi.org/10.48550/arxiv.0911.0290

2009-01-01

Abstract:A logarithmic type Harnack inequality is established for the semigroup of solutions to a stochastic differential equation in Hilbert spaces with non-additive noise. As applications, the strong Feller property as well as the entropy-cost inequality for the semigroup are derived with respect to the corresponding distance (cost function).
Empirical Bernstein in smooth Banach spaces

Diego Martinez-Taboada,Aaditya Ramdas

2024-09-10

Abstract:Existing concentration bounds for bounded vector-valued random variables include extensions of the scalar Hoeffding and Bernstein inequalities. While the latter is typically tighter, it requires knowing a bound on the variance of the random variables. We derive a new vector-valued empirical Bernstein inequality, which makes use of an empirical estimator of the variance instead of the true variance. The bound holds in 2-smooth separable Banach spaces, which include finite dimensional Euclidean spaces and separable Hilbert spaces. The resulting confidence sets are instantiated for both the batch setting (where the sample size is fixed) and the sequential setting (where the sample size is a stopping time). The confidence set width asymptotically exactly matches that achieved by Bernstein in the leading term. The method and supermartingale proof technique combine several tools of Pinelis (1994) and Waudby-Smith and Ramdas (2024).

Statistics Theory
Non-central limit of densities of some functionals of Gaussian processes

Solesne Bourguin,Thanh Dang,Yaozhong Hu

2024-08-13

Abstract:We establish the convergence of the densities of a sequence of nonlinear functionals of an underlying Gaussian process to the density of a Gamma distribution. The key idea of our work is a new density formula for random variables in the setting of Markov diffusion generators, which yields a special representation for the density of a Gamma distribution. Via this representation, we are able to provide precise estimates on the distance between densities while developing the techniques of Malliavin calculus and Stein's method suitable to Gamma approximation at the density level. We first focus our study on the case of random variables living in a fixed Wiener chaos of an even order for which the bound for the difference of the densities can be dominated by a linear combination of moments up to order four. We then study the case of general Gaussian functionals with possibly infinite chaos expansion. Finally, we provide two applications of our results, namely the limit of random variables living in the second Wiener chaos, and the limit of sums of independent Pareto distributed random variables.

Probability

Bounding Hellinger Distance with Stein's Method

On Bernstein Type Inequalities for Stochastic Integrals of Multivariate Point Processes

Bounding the -Distance Between One-Dimensional Continuous and Discrete Distributions via Stein's Method

Efficient Concentration with Gaussian Approximation

Quantitative and stable limits of high-frequency statistics of Lévy processes: a Stein's method approach

One-dimensional Stein's method with bespoke derivatives

Maximum interpoint distance of high-dimensional random vectors

Stein's density method for multivariate continuous distributions

A minimum Hellinger distance estimator for stochastic differential equations: An application to statistical inference for continuous time interest rate models

Total variation bound for Hadwiger's functional using Stein's method

Bounds for distributional approximation in the multivariate delta method by Stein's method

Multidimensional Stein's method for Gamma approximation

Stein's method, smoothing and functional approximation

Error Bounds on a Mixed Entropy Inequality

Stein's Method for Spatial Random Graphs

Gaussian random field approximation via Stein's method with applications to wide random neural networks

Sharp supremum and Hölder bounds for stochastic integrals indexed by a parameter

New bounds for normal approximation on product spaces with applications to monochromatic edges, random sums and an infinite de Jong CLT

Log-Harnack Inequality for Stochastic Differential Equations in Hilbert Spaces and Its Consequences

Empirical Bernstein in smooth Banach spaces

Non-central limit of densities of some functionals of Gaussian processes