A simple polynomial-time approximation algorithm for the total variation distance between two product distributions

Weiming Feng,Heng Guo,Mark Jerrum,Jiaheng Wang

DOI: https://doi.org/10.46298/theoretics.23.7

2023-06-12

Abstract:We give a simple polynomial-time approximation algorithm for the total variation distance between two product distributions.

Data Structures and Algorithms,Methodology

What problem does this paper attempt to address?

Approximation of Sums of Locally Dependent Random Variables via Perturbation of Stein Operator

Zhonggen Su,Xiaolin Wang

2022-01-01

Abstract: Let $(X_i, i\in J)$ be a family of locally dependent non-negative integer-valued random variables, and consider the sum $W=\sum_{i\in J}X_i$. We establish general error upper bounds for $d_{TV}(W, M)$ using the three-parametric perturbation form of Stein operator, where the target variable $M$ is either the mixture of Poisson distribution and binomial or negative binomial distribution, and the three parameters are delicately determined by guaranteeing $M$ and $W$ are identical in the first three factorial cumulants. As applications, we discuss in detail the classic local dependence structure examples like $(1, 1)$-runs, ($k_{1},k_{2}$)-runs and $k$-runs, and attain $O(|J|^{-1})$ error bounds under some special cases by exactly figuring out the smoothness of the conditional distributions of $W$ upon partial sums. The obtained result is to some extent a significant improvement of Pek\"{o}z [Bernoulli, 19 (2013)] and Upadhye, et al. [Bernoulli, 23 (2017)].
Dynamic Time Warping under Product Quantization, with Applications to Time-Series Data Similarity Search

Haowen Zhang,Yabo Dong,Jing Li,Duanqing Xu

DOI: https://doi.org/10.1109/jiot.2021.3132017

IF: 10.6

2021-01-01

IEEE Internet of Things Journal

Abstract:The similarity search on sensor data generated by a myriad of sensing devices is a frequently encountered problem in the era of the Internet of Things (IoT). This sensor data generally appear in the form of time series, a temporally ordered sequence of real numbers obtained regularly in time. It has been widely accepted that the dynamic time warping (DTW) currently is the most prevalent similarity measure in the time-series mining community, mainly due to its flexibility and broad applicability. However, calculating DTW between two time series has quadratic time complexity, leading to unsatisfactory efficiency when performing the similarity search over the large time-series data set. The main contribution of this article is to propose a method called product quantization (PQ)-based DTW (PQDTW) for fast time-series approximate similarity search under DTW. The PQ, a well-known approximate nearest neighbor search approach, is used in PQDTW. Nevertheless, the conventional PQ is developed with the Euclidean distance and is not designed for DTW. To solve this problem, the DTW barycenter averaging (DBA) technique is utilized to adapt the PQ for DTW before using it. We employ PQDTW along with the filter-and-refine framework to efficiently and accurately perform the time-series similarity search. Our method can reasonably reduce many DTW computations in the filtering phase; thus, the query process is accelerated. We compare PQDTW with related popular algorithms using public time-series data sets. Experimental results verify that the proposal achieves the best tradeoff between query efficiency and retrieval accuracy compared to the competitors.
Approximation of the Tail Probability of Randomly Weighted Sums and Applications

Yi Zhang,Xinmei Shen,Chengguo Weng

DOI: https://doi.org/10.1016/j.spa.2008.03.004

IF: 1.43

2009-01-01

Stochastic Processes and their Applications

Abstract:Consider the problem of approximating the tail probability of randomly weighted sums ∑i=1nΘiXi and their maxima, where {Xi,i≥1} is a sequence of identically distributed but not necessarily independent random variables from the extended regular variation class, and {Θi,i≥1} is a sequence of nonnegative random variables, independent of {Xi,i≥1} and satisfying certain moment conditions. Under the assumption that {Xi,i≥1} has no bivariate upper tail dependence along with some other mild conditions, this paper establishes the following asymptotic relations: Pr(max1≤k≤n∑i=1kΘiXi>x)∼Pr(∑i=1nΘiXi>x)∼∑i=1nPr(ΘiXi>x), and Pr(max1≤k<∞∑i=1kΘiXi>x)∼Pr(∑i=1∞ΘiXi+>x)∼∑i=1∞Pr(ΘiXi>x), as x→∞. In doing so, no assumption is made on the dependence structure of the sequence {Θi,i≥1}.
Total Variation Distance for Product Distributions is $\#\mathsf{P}$-Complete

Arnab Bhattacharyya,Sutanu Gayen,Kuldeep S. Meel,Dimitrios Myrisiotis,A. Pavan,N. V. Vinodchandran

2024-05-14

Abstract:We show that computing the total variation distance between two product distributions is $\#\mathsf{P}$-complete. This is in stark contrast with other distance measures such as Kullback-Leibler, Chi-square, and Hellinger, which tensorize over the marginals leading to efficient algorithms.

Computational Complexity
A 3-approximation algorithm for two-dimensional bin packing

Guochuan Zhang

DOI: https://doi.org/10.1016/j.orl.2004.04.004

IF: 1.151

2005-01-01

Operations Research Letters

Abstract:In the classical two-dimensional bin packing problem one is asked to pack a set of rectangular items, without overlap and without any rotation, into the minimum number of identical square bins. We give an approximation algorithm with absolute worst-case ratio of 3.
Discriminative Estimation of Total Variation Distance: A Fidelity Auditor for Generative Data

Lan Tao,Shirong Xu,Chi-Hua Wang,Namjoon Suh,Guang Cheng

2024-05-24

Abstract:With the proliferation of generative AI and the increasing volume of generative data (also called as synthetic data), assessing the fidelity of generative data has become a critical concern. In this paper, we propose a discriminative approach to estimate the total variation (TV) distance between two distributions as an effective measure of generative data fidelity. Our method quantitatively characterizes the relation between the Bayes risk in classifying two distributions and their TV distance. Therefore, the estimation of total variation distance reduces to that of the Bayes risk. In particular, this paper establishes theoretical results regarding the convergence rate of the estimation error of TV distance between two Gaussian distributions. We demonstrate that, with a specific choice of hypothesis class in classification, a fast convergence rate in estimating the TV distance can be achieved. Specifically, the estimation accuracy of the TV distance is proven to inherently depend on the separation of two Gaussian distributions: smaller estimation errors are achieved when the two Gaussian distributions are farther apart. This phenomenon is also validated empirically through extensive simulations. In the end, we apply this discriminative estimation method to rank fidelity of synthetic image data using the MNIST dataset.

Machine Learning
Improved bounds for the total variation distance between stochastic polynomials

Egor Kosov,Anastasia Zhukova

2023-08-04

Abstract:The paper studies upper bounds for the total variation distance between two polynomials of a special form in random vectors satisfying the Doeblin-type condition. Our approach is based on the recent results concerning Nikolskii--Besov-type smoothness of distribution densities of polynomials in logarithmically concave random vectors. The main results of the paper improve previously obtained estimates of Nourdin--Poly and Bally--Caramellino.

Probability
Total Variation Distance Meets Probabilistic Inference

Arnab Bhattacharyya,Sutanu Gayen,Kuldeep S. Meel,Dimitrios Myrisiotis,A. Pavan,N. V. Vinodchandran

2024-07-01

Abstract:In this paper, we establish a novel connection between total variation (TV) distance estimation and probabilistic inference. In particular, we present an efficient, structure-preserving reduction from relative approximation of TV distance to probabilistic inference over directed graphical models. This reduction leads to a fully polynomial randomized approximation scheme (FPRAS) for estimating TV distances between same-structure distributions over any class of Bayes nets for which there is an efficient probabilistic inference algorithm. In particular, it leads to an FPRAS for estimating TV distances between distributions that are defined over a common Bayes net of small treewidth. Prior to this work, such approximation schemes only existed for estimating TV distances between product distributions. Our approach employs a new notion of $partial$ couplings of high-dimensional distributions, which might be of independent interest.

Data Structures and Algorithms,Computational Complexity,Discrete Mathematics,Machine Learning
Approximating the Earth Mover's Distance between sets of geometric objects

Marc van Kreveld,Frank Staals,Amir Vaxman,Jordi Vermeulen

DOI: https://doi.org/10.48550/arXiv.2104.08136

2023-02-17

Abstract:Given two distributions $P$ and $S$ of equal total mass, the Earth Mover's Distance measures the cost of transforming one distribution into the other, where the cost of moving a unit of mass is equal to the distance over which it is moved. We give approximation algorithms for the Earth Mover's Distance between various sets of geometric objects. We give a $(1 + \varepsilon)$-approximation when $P$ is a set of weighted points and $S$ is a set of line segments, triangles or $d$-dimensional simplices. When $P$ and $S$ are both sets of line segments, sets of triangles or sets of simplices, we give a $(1 + \varepsilon)$-approximation with a small additive term. All algorithms run in time polynomial in the size of $P$ and $S$, and actually calculate the transport plan (that is, a specification of how to move the mass), rather than just the cost. To our knowledge, these are the first combinatorial algorithms with a provable approximation ratio for the Earth Mover's Distance when the objects are continuous rather than discrete points.

Computational Geometry
On the tensorization of the variational distance

Aryeh Kontorovich

2024-10-02

Abstract:If one seeks to estimate the total variation between two product measures $||P^\otimes_{1:n}-Q^\otimes_{1:n}||$ in terms of their marginal TV sequence $\delta=(||P_1-Q_1||,||P_2-Q_2||,\ldots,||P_n-Q_n||)$, then trivial upper and lower bounds are provided by$ ||\delta||_\infty \le ||P^\otimes_{1:n}-Q^\otimes_{1:n}||\le||\delta||_1$. We improve the lower bound to $||\delta||_2\lesssim||P^\otimes_{1:n}-Q^\otimes_{1:n}||$, thereby reducing the gap between the upper and lower bounds from $\sim n$ to $\sim\sqrt $. Furthermore, we show that {\em any} estimate on $||P^\otimes_{1:n}-Q^\otimes_{1:n}||$ expressed in terms of $\delta$ must necessarily exhibit a gap of $\sim\sqrt n$ between the upper and lower bounds in the worst case, establishing a sense in which our estimate is optimal. Finally, we identify a natural class of distributions for which $||\delta||_2$ approximates the TV distance up to absolute multiplicative constants.

Probability
Approximation to Distribution of Product of Random Variables Using Orthogonal Polynomials for Lognormal Density

Zhong Zheng,Lu Wei,Jyri Hämäläinen,Olav Tirkkonen

DOI: https://doi.org/10.1109/LCOMM.2012.101712.122141

2012-07-16

Abstract:We derive a closed-form expression for the orthogonal polynomials associated with the general lognormal density. The result can be utilized to construct easily computable approximations for probability density function of a product of random variables, when the considered variates are either independent or correlated. As an example, we have calculated the approximative distribution for the product of Nakagami-m variables. Simulations indicate that accuracy of the proposed approximation is good with small cross-correlations under light fading condition.

Information Theory
Three-Parameter Approximations of Sums of Locally Dependent Random Variables via Stein's Method

Zhonggen Su,Xiaolin Wang

DOI: https://doi.org/10.1007/s11425-023-2255-1

2024-09-06

Abstract:Let $\{X_{i}, i\in J\}$ be a family of locally dependent non-negative integer-valued random variables with finite expectations and variances. We consider the sum $W=\sum_{i\in J}X_i$ and use Stein's method to establish general upper error bounds for the total variation distance $d_{TV}(W, M)$, where $M$ represents a three-parameter random variable. As a direct consequence, we obtain a discretized normal approximation for $W$. As applications, we study in detail four well-known examples, which are counting vertices of all edges point inward, birthday problem, counting monochromatic edges in uniformly colored graphs, and triangles in the Erdős-Rényi random graph. Through delicate analysis and computations, we obtain sharper upper error bounds than existing results.

Probability
Total variation distance between a jump-equation and its Gaussian approximation

Vlad Bally,Yifeng Qin

DOI: https://doi.org/10.48550/arXiv.2109.11208

2023-10-16

Abstract:We deal with stochastic differential equations with jumps. In order to obtain an accurate approximation scheme, it is usual to replace the "small jumps" by a Brownian motion. In this paper, we prove that for every fixed time $t$, the approximate random variable $X^\varepsilon\_t$ converges to the original random variable $X\_t$ in total variation distance and we estimate the error. We also give an estimate of the distance between the densities of the laws of the two random variables. These are done by using some integration by parts techniques in Malliavin calculus.

Probability
Total variation distance and compound poisson approximations for random sums

Stathis Chadjiconstantinidis

DOI: https://doi.org/10.1080/07362994.2024.2321348

2024-03-10

Stochastic Analysis and Applications

Abstract:We derive new upper bounds for the total variation distance between compound Poisson distributions as well as between a random sum and a compound Poisson distribution, and as a result we also obtain upper bounds for the total variation distance between compound Poisson distributions and a sum of independent random variables. These bounds are generalizations and refinements of some well-known bounds in the literature. We also derive upper bounds for the total variation distance between negative binomial distributions of order k and between the negative binomial distributions of order k and compound Poisson distributions. Upper bounds for the total variation distances between the number of success runs of length k in binary Markovian trials and its limiting distributions for several enumeration schemes, are also given.

mathematics, applied,statistics & probability
A Fully Polynomial-Time Approximation Scheme for Approximating a Sum of Random Variables

Jian Li,Tianlin Shi

DOI: https://doi.org/10.1016/j.orl.2014.02.004

IF: 1.151

2014-01-01

Operations Research Letters

Abstract:Given n independent integer-valued random variables X-1, X-2, . . . , X-n and an integer C, we study the fundamental problem of computing the probability that the sum X = X-1 + X-2 + . . . + X-n is at most C. We assume that each random variable xi is implicitly given by an oracle O-i, which given two input integers n(1), n(2) returns the probability of n(1) <= X-i <= n(2). We give the first deterministic fully polynomial-time approximation scheme (FPTAS) to estimate the probability up to a relative error of 1 +/- epsilon. Our algorithm is based on the technique for approximately counting knapsack solutions, developed in Gopalan et al. (2011). (C) 2014 Elsevier B.V. All rights reserved.
A deterministic near-linear time approximation scheme for geometric transportation

Emily Fox,Jiashuai Lu

2023-09-28

Abstract:Given a set of points $P = (P^+ \sqcup P^-) \subset \mathbb{R}^d$ for some constant $d$ and a supply function $\mu:P\to \mathbb{R}$ such that $\mu(p) > 0~\forall p \in P^+$, $\mu(p) < 0~\forall p \in P^-$, and $\sum_{p\in P}{\mu(p)} = 0$, the geometric transportation problem asks one to find a transportation map $\tau: P^+\times P^-\to \mathbb{R}_{\ge 0}$ such that $\sum_{q\in P^-}{\tau(p, q)} = \mu(p)~\forall p \in P^+$, $\sum_{p\in P^+}{\tau(p, q)} = -\mu(q)~ \forall q \in P^-$, and the weighted sum of Euclidean distances for the pairs $\sum_{(p,q)\in P^+\times P^-}\tau(p, q)\cdot ||q-p||_2$ is minimized. We present the first deterministic algorithm that computes, in near-linear time, a transportation map whose cost is within a $(1 + \varepsilon)$ factor of optimal. More precisely, our algorithm runs in $O(n\varepsilon^{-(d+2)}\log^5{n}\log{\log{n}})$ time for any constant $\varepsilon > 0$. Surprisingly, our result is not only a generalization of a bipartite matching one to arbitrary instances of geometric transportation, but it also reduces the running time for all previously known $(1 + \varepsilon)$-approximation algorithms, randomized or deterministic, even for geometric bipartite matching. In particular, we give the first $(1 + \varepsilon)$-approximate deterministic algorithm for geometric bipartite matching and the first $(1 + \varepsilon)$-approximate deterministic or randomized algorithm for geometric transportation with no dependence on $d$ in the exponent of the running time's polylog. As an additional application of our main ideas, we also give the first randomized near-linear $O(\varepsilon^{-2} m \log^{O(1)} n)$ time $(1 + \varepsilon)$-approximation algorithm for the uncapacitated minimum cost flow (transshipment) problem in undirected graphs with arbitrary real edge costs.

Computational Geometry,Data Structures and Algorithms
Greedy Algorithms for Optimal Distribution Approximation

Bernhard C. Geiger,Georg Böcherer

DOI: https://doi.org/10.3390/e18070262

2016-01-22

Abstract:The approximation of a discrete probability distribution $\mathbf{t}$ by an $M$-type distribution $\mathbf{p}$ is considered. The approximation error is measured by the informational divergence $\mathbb{D}(\mathbf{t}\Vert\mathbf{p})$, which is an appropriate measure, e.g., in the context of data compression. Properties of the optimal approximation are derived and bounds on the approximation error are presented, which are asymptotically tight. It is shown that $M$-type approximations that minimize either $\mathbb{D}(\mathbf{t}\Vert\mathbf{p})$, or $\mathbb{D}(\mathbf{p}\Vert\mathbf{t})$, or the variational distance $\Vert\mathbf{p}-\mathbf{t}\Vert_1$ can all be found by using specific instances of the same general greedy algorithm.

Information Theory
A polynomialâ€�time approximation scheme for the maximal overlap of two independent ErdÅ‘sâ€“RÃ©nyiÂ graphs

Jian Ding,Hang Du,Shuyang Gong

DOI: https://doi.org/10.1002/rsa.21212

2024-03-09

Random Structures and Algorithms

Abstract:For two independent ErdÅ‘sâ€“RÃ©nyi graphs G(n,p) , we study the maximal overlap (i.e., the number of common edges) of these two graphs over all possible vertex correspondence. We present a polynomialâ€�time algorithm which finds a vertex correspondence whose overlap approximates the maximal overlap up to a multiplicative factor that is arbitrarily close to 1. As a byâ€�product, we prove that the maximal overlap is asymptotically n2Î±âˆ’1 for p=nâˆ’Î± with some constant Î±âˆˆ(1/2,1) .

mathematics, applied,computer science, software engineering
Polynomial-time Algorithms for Computing Distances of Fuzzy Transition Systems.

Taolue Chen,Tingting Han,Yongzhi Cao

DOI: https://doi.org/10.1016/j.tcs.2018.03.002

IF: 1.002

2018-01-01

Theoretical Computer Science

Abstract:Behaviour distances to measure the resemblance of two states in a (nondeterministic) fuzzy transition system have been proposed recently in literature. Such a distance, defined as a pseudo-ultrametric over the state space of the model, provides a quantitative analogue of bisimilarity. In this paper, we focus on the problem of computing these distances. We first extend the definition of the pseudo-ultrametric by introducing discount such that the discounting factor being equal to 1 captures the original definition. We then provide polynomial-time algorithms to calculate the behavioural distances, in both the non-discounted and the discounted setting. The algorithm is strongly polynomial in the former case.
On the Analysis of Average Time Complexity of Estimation of Distribution Algorithms

Tianshi Chen,Ke Tang,Guoliang Chen,Xin Yao

DOI: https://doi.org/10.1109/cec.2007.4424506

2007-01-01

Abstract:Estimation of Distribution Algorithm (EDA) is a well-known stochastic optimization technique. The average time complexity is a crucial criterion that measures the performance of the stochastic algorithms. In the past few years, various kinds of EDAs have been proposed, but the related theoretical study on the time complexity of these algorithms is relatively few. This paper analyzed the time complexity of two early versions of EDA, the Univariate Marginal Distribution Algorithm (UMDA) and the Incremental UMDA (IUMDA). We generalize the concept of convergence to convergence time, and manage to estimate the upper bound of the mean First Hitting Times (FHTs) of UMDA (IUMDA) on a well-known pseudo-modular function, which is frequently studied in the field of genetic algorithms. Our analysis shows that UMDA (IUMDA) has O(n) behaviors on the pseudo-modular function. In addition, we analyze the mean FHT of IUMDA on a hard problem. Our result shows that IUMDA may spend exponential generations to find the global optimum. This is the first time that the mean first hitting times of UMDA (IUMDA) are theoretically studied.

A simple polynomial-time approximation algorithm for the total variation distance between two product distributions

Approximation of Sums of Locally Dependent Random Variables via Perturbation of Stein Operator

Dynamic Time Warping under Product Quantization, with Applications to Time-Series Data Similarity Search

Approximation of the Tail Probability of Randomly Weighted Sums and Applications

Total Variation Distance for Product Distributions is $\#\mathsf{P}$-Complete

A 3-approximation algorithm for two-dimensional bin packing

Discriminative Estimation of Total Variation Distance: A Fidelity Auditor for Generative Data

Improved bounds for the total variation distance between stochastic polynomials

Total Variation Distance Meets Probabilistic Inference

Approximating the Earth Mover's Distance between sets of geometric objects

On the tensorization of the variational distance

Approximation to Distribution of Product of Random Variables Using Orthogonal Polynomials for Lognormal Density

Three-Parameter Approximations of Sums of Locally Dependent Random Variables via Stein's Method

Total variation distance between a jump-equation and its Gaussian approximation

Total variation distance and compound poisson approximations for random sums

A Fully Polynomial-Time Approximation Scheme for Approximating a Sum of Random Variables

A deterministic near-linear time approximation scheme for geometric transportation

Greedy Algorithms for Optimal Distribution Approximation

A polynomialâ€�time approximation scheme for the maximal overlap of two independent ErdÅ‘sâ€“RÃ©nyiÂ graphs

Polynomial-time Algorithms for Computing Distances of Fuzzy Transition Systems.

On the Analysis of Average Time Complexity of Estimation of Distribution Algorithms