Range-Renewal Structure of I.I.D. Samplings

lihong wang,jiansheng xie

2013-01-01

Abstract: In this note the range-renewal structure of general independent and identically distributed samplings is discussed, which is a natural extension of the result in Chen et al (arXiv:1305.1829).

What problem does this paper attempt to address?

CLT, MDP and LDP for Range-Renewals of I.I.D.Samplings from an Infinite Discrete Distribution

Xin-Xing Chen,Jian-Sheng Xie,Min-Zhi Zhao

DOI: https://doi.org/10.48550/arxiv.2211.08815

2022-01-01

Abstract: Let $R_n$ be the number of distinct values of the $n$ simple samples from an infinite discrete distribution. In 1960 Bahadure proved $\displaystyle \lim_{n\to \infty} \frac{R_n}{\Enum R_n}=1$ in probability; Chen et al. proved the limit in the sense of almost sure convergence, along with other results. In this note we present results of CLT, MDP and LDP for $R_n$ under mild conditions.
Range-Renewal Speed and Entropy for I.I.D Models

xinxing chen,jiansheng xie,jiangang ying

2013-01-01

Abstract: In this note the relation between the range-renewal speed and entropy for i.i.d. models is discussed.
On Increments of Sums of Independent Non-Identically Distributed Random-Variables

ZY LIN

1988-01-01

Abstract:The paper gives some theorems about the increments of partial sums of independent, but not necessarily independent identically distributed random variables. The main results are ideal and corresponding to those for i. i. d. random variables. Therefore these sharp results of Hanson and Russo (1983) answer some of their questions.
Range-Renewal Processes: SLLN, Power Law and Beyonds

xinxing chen,jiansheng xie,jiangang ying

2013-01-01

Abstract: Given $n$ samples of a regular discrete distribution $\pi$, we prove in this article first a serial of SLLNs results (of Dvoretzky and Erd\"{o}s' type) which implies a typical power law when $\pi$ is heavy-tailed. Constructing a (random) graph from the ordered $n$ samples, we can establish other laws for the degree-distribution of the graph. The phenomena of small world is also discussed.
Range-Renewal Structure of Transient Simple Random Walk

Jian-Sheng Xie,Yuan-Yuan Xu

DOI: https://doi.org/10.1016/j.spl.2013.05.031

IF: 0.718

2013-01-01

Statistics & Probability Letters

Abstract:Dvoretzky and Erdös (1950) proved limn→+∞Rnn=γ almost surely for the simple symmetric random walk on Zd (d≥3) with Rn being the number of distinct cites visited in the time interval [0,n) and γ=γd being the escape rate. Following Derriennic’s approach, we prove further in this note that limn→+∞Rn,kRn=γ(1−γ)k−1,limn→+∞Rn,kRn,k+=γ,k=1,2,⋯ almost surely with Rn,k (resp. Rn,k+) being the number of distinct cites visited exactly (resp. at least) k times in the time interval [0,n). Such results are generalized to transient simple random walks on discrete groups.
IID Sampling from Intractable Distributions

Sourabh Bhattacharya

DOI: https://doi.org/10.48550/arXiv.2107.05956

2021-12-15

Abstract:We propose a novel methodology for drawing iid realizations from any target distribution on the Euclidean space with arbitrary dimension. No assumption of compact support is necessary for the validity of our theory and method. Our idea is to construct an appropriate infinite sequence of concentric closed ellipsoids, represent the target distribution as an infinite mixture on the central ellipsoid and the ellipsoidal annuli, and to construct efficient perfect samplers for the mixture components. In contrast with most of the existing works on perfect sampling, ours is not only a theoretically valid method, it is practically applicable to all target distributions on any dimensional Euclidean space and very much amenable to parallel computation. We validate the practicality and usefulness of our methodology by generating 10000 iid realizations from the standard distributions such as normal, Student's t with 5 degrees of freedom and Cauchy, for dimensions d = 1, 5, 10, 50, 100, as well as from a 50-dimensional mixture normal distribution. The implementation time in all the cases are very reasonable, and often less than a minute in our parallel implementation. The results turned out to be highly accurate. We also apply our method to draw 10000 iid realizations from the posterior distributions associated with the well-known Challenger data, a Salmonella data and the 160-dimensional challenging spatial example of the radionuclide count data on Rongelap Island. Again, we are able to obtain quite encouraging results with very reasonable computing time.

Computation,Methodology
Extension of Donsker's Invariance Principle with Incomplete Partial-Sum Process

Jingwei Liu

DOI: https://doi.org/10.48550/arXiv.1912.07215

2019-12-16

Probability

Abstract:Based on deleting-item central limit theory, the classical Donsker's theorem of partial-sum process of independent and identically distributed (i.i.d.) random variables is extended to incomplete partial-sum process. The incomplete partial-sum process Donsker's invariance principles are constructed and derived for general partial-sum process of i.i.d random variables and empirical process respectively, they are not only the extension of functional central limit theory, but also the extension of deleting-item central limit theory. Our work enriches the random elements structure of weak convergence.
Near-linear time samplers for matroid independent sets with applications

Xiaoyu Chen,Heng Guo,Xinyuan Zhang,Zongrui Zou

2023-09-27

Abstract:We give a $\widetilde{O}(n)$ time almost uniform sampler for independent sets of a matroid, whose ground set has $n$ elements and is given by an independence oracle. As a consequence, one can sample connected spanning subgraphs of a given graph $G=(V,E)$ in $\widetilde{O}(|E|)$ time. This leads to improved running time on estimating all-terminal network reliability. Furthermore, we generalise this near-linear time sampler to the random cluster model with $q\le 1$.

Data Structures and Algorithms,Probability
Independent Range Sampling on Interval Data (Longer Version)

Daichi Amagata

2024-05-22

Abstract:Many applications require efficient management of large sets of intervals because many objects are associated with intervals (e.g., time and price intervals). In such interval management systems, range search is a primitive operator for retrieving and analysis tasks. As dataset sizes are growing nowadays, range search results are also becoming larger, which may overwhelm users and incur long computation time. Because applications are usually satisfied with a subset of the result set, it is desirable to efficiently obtain only small samples from the result set.We therefore address the problem of independent range sampling on interval data, which outputs $s$ random samples that overlap a given query interval and are independent of the samples of all previous queries. To efficiently solve this problem theoretically and practically, we propose a variant of an interval tree, namely the augmented interval tree (or AIT), and we show that there exists an exact algorithm that needs $O(n \log n)$ space and $O(\log^{2} n + s)$ time, where $n$ is the dataset size. The simple structure of an AIT provides flexible extensions: (i) its time and space complexities respectively become $O(\log^{2} n + s)$ expected and $O(n)$ by bucketing intervals and (ii) it can deal with weighted intervals and outputs $s$ weighted random samples in $O(\log^{2} n+s\log n)$ time. We conduct extensive experiments on real datasets, and the results demonstrate that our algorithms significantly outperform competitors.

Databases
Achieving Efficiency in Black Box Simulation of Distribution Tails with Self-structuring Importance Samplers

Anand Deo,Karthyek Murthy

2023-07-08

Abstract:This paper presents a novel Importance Sampling (IS) scheme for estimating distribution tails of performance measures modeled with a rich set of tools such as linear programs, integer linear programs, piecewise linear/quadratic objectives, feature maps specified with deep neural networks, etc. The conventional approach of explicitly identifying efficient changes of measure suffers from feasibility and scalability concerns beyond highly stylized models, due to their need to be tailored intricately to the objective and the underlying probability distribution. This bottleneck is overcome in the proposed scheme with an elementary transformation which is capable of implicitly inducing an effective IS distribution in a variety of models by replicating the concentration properties observed in less rare samples. This novel approach is guided by developing a large deviations principle that brings out the phenomenon of self-similarity of optimal IS distributions. The proposed sampler is the first to attain asymptotically optimal variance reduction across a spectrum of multivariate distributions despite being oblivious to the specifics of the underlying model. Its applicability is illustrated with contextual shortest path and portfolio credit risk models informed by neural networks

Machine Learning,Probability,Methodology
SINGLE-RECORD RANDOM INCREMENTAL DYNAMIC ANALYSIS

LU Da-gang,YU Xiao-hui,WANG Guang-yuan

2010-01-01

Abstract:The conventional Incremental Dynamic Analysis (IDA) method doesn't take into account the inherent randomness in structures.In this paper,a new IDA method considering structural random properties is put forward,which combines a new point estimate method based on Nataf transformation with the single-record IDA method.The new approach is applied to a R.C.frame structure,for which the random IDA curves are obtained.It is demonstrated through a numerical example that the new IDA method can be efficiently used to investigate the effect of structural random properties on IDA results.
A LIL for Independent Non-Identically Distributed Random Variables in Banach Space and Its Applications

Liu WeiDong,Fu KeAng,Zhang LiXin

DOI: https://doi.org/10.1007/s11425-007-0197-y

2008-01-01

Science in China Series A Mathematics

Abstract:In this paper, we prove a general law of the iterated logarithm (LIL) for independent non-identically distributed B-valued random variables. As an interesting application, we obtain the law of the iterated logarithm for the empirical covariance of Hilbertian autoregressive processes.
Risk Bounds for Infinitely Divisible Distribution

Chao Zhang,Dacheng Tao

DOI: https://doi.org/10.48550/arxiv.1202.3774

2011-01-01

Abstract:In this paper, we study the risk bounds for samples independently drawn from an infinitely divisible (ID) distribution. In particular, based on a martingale method, we develop two deviation inequalities for a sequence of random variables of an ID distribution with zero Gaussian component. By applying the deviation inequalities, we obtain the risk bounds based on the covering number for the ID distribution. Finally, we analyze the asymptotic convergence of the risk bound derived from one of the two deviation inequalities and show that the convergence rate of the bound is faster than the result for the generic i.i.d. empirical process (Mendelson, 2003).
Optimal scaling of the independence sampler: Theory and practice

Clement Lee,Peter Neal

DOI: https://doi.org/10.3150/16-bej908

IF: 1.5

2018-08-01

Bernoulli

Abstract:The independence sampler is one of the most commonly used MCMC algorithms usually as a component of a Metropolis-within-Gibbs algorithm. The common focus for the independence sampler is on the choice of proposal distribution to obtain an as high as possible acceptance rate. In this paper, we have a somewhat different focus concentrating on the use of the independence sampler for updating augmented data in a Bayesian framework where a natural proposal distribution for the independence sampler exists. Thus, we concentrate on the proportion of the augmented data to update to optimise the independence sampler. Generic guidelines for optimising the independence sampler are obtained for independent and identically distributed product densities mirroring findings for the random walk Metropolis algorithm. The generic guidelines are shown to be informative beyond the narrow confines of idealised product densities in two epidemic examples.

statistics & probability
Range-Renewal Processes: SLLNs and Power Laws

Chen Xinxing,Xie Jiansheng,Ying Jiangang

DOI: https://doi.org/10.1007/s11401-022-0305-x

2022-01-01

Abstract:Given n samples (viewed as an n-tuple) of a γ-regular discrete distribution π, in this article the authors concern with the weighted and unweighted graphs induced by the n samples. They first prove a series of SLLN results (of Dvoretzky-Erdös’ type). Then they show that the vertex weights of the graphs under investigation obey asymptotically power law distributions with exponent 1 + γ. They also give a conjecture that the degrees of unweighted graphs would exhibit asymptotically power law distributions with constant exponent 2. This exponent is obviously independent of the parameter γ ∈ (0, 1), which is a surprise to us at first sight.
Subset Sampling and Its Extensions

Jinchao Huang,Sibo Wang

2023-07-21

Abstract:This paper studies the \emph{subset sampling} problem. The input is a set $\mathcal{S}$ of $n$ records together with a function $\textbf{p}$ that assigns each record $v\in\mathcal{S}$ a probability $\textbf{p}(v)$. A query returns a random subset $X$ of $\mathcal{S}$, where each record $v\in\mathcal{S}$ is sampled into $X$ independently with probability $\textbf{p}(v)$. The goal is to store $\mathcal{S}$ in a data structure to answer queries efficiently. If $\mathcal{S}$ fits in memory, the problem is interesting when $\mathcal{S}$ is dynamic. We develop a dynamic data structure with $\mathcal{O}(1+\mu_{\mathcal{S}})$ expected \emph{query} time, $\mathcal{O}(n)$ space and $\mathcal{O}(1)$ amortized expected \emph{update}, \emph{insert} and \emph{delete} time, where $\mu_{\mathcal{S}}=\sum_{v\in\mathcal{S}}\textbf{p}(v)$. The query time and space are optimal. If $\mathcal{S}$ does not fit in memory, the problem is difficult even if $\mathcal{S}$ is static. Under this scenario, we present an I/O-efficient algorithm that answers a \emph{query} in $\mathcal{O}\left((\log^*_B n)/B+(\mu_\mathcal{S}/B)\log_{M/B} (n/B)\right)$ amortized expected I/Os using $\mathcal{O}(n/B)$ space, where $M$ is the memory size, $B$ is the block size and $\log^*_B n$ is the number of iterative $\log_2(.)$ operations we need to perform on $n$ before going below $B$. In addition, when each record is associated with a real-valued key, we extend the \emph{subset sampling} problem to the \emph{range subset sampling} problem, in which we require that the keys of the sampled records fall within a specified input range $[a,b]$. For this extension, we provide a solution under the dynamic setting, with $\mathcal{O}(\log n+\mu_{\mathcal{S}\cap[a,b]})$ expected \emph{query} time, $\mathcal{O}(n)$ space and $\mathcal{O}(\log n)$ amortized expected \emph{update}, \emph{insert} and \emph{delete} time.

Data Structures and Algorithms,Databases
A renewal theory approach to IBD sharing

Shai Carmi,Peter R. Wilton,John Wakeley,Itsik Pe’er

DOI: https://doi.org/10.1016/j.tpb.2014.08.002

IF: 1.514

2014-11-01

Theoretical Population Biology

Abstract:A long genomic segment inherited by a pair of individuals from a single, recent common ancestor is said to be identical-by-descent (IBD). Shared IBD segments have numerous applications in genetics, from demographic inference to phasing, imputation, pedigree reconstruction, and disease mapping. Here, we provide a theoretical analysis of IBD sharing under Markovian approximations of the coalescent with recombination. We describe a general framework for the IBD process along the chromosome under the Markovian models (SMC/SMC'), as well as introduce and justify a new model, which we term the renewal approximation, under which lengths of successive segments are independent. Then, considering the infinite-chromosome limit of the IBD process, we recover previous results (for SMC) and derive new results (for SMC') for the mean number of shared segments longer than a cutoff and the fraction of the chromosome found in such segments. We then use renewal theory to derive an expression (in Laplace space) for the distribution of the number of shared segments and demonstrate implications for demographic inference. We also compute (again, in Laplace space) the distribution of the fraction of the chromosome in shared segments, from which we obtain explicit expressions for the first two moments. Finally, we generalize all results to populations with a variable effective size.

genetics & heredity,ecology,evolutionary biology,mathematical & computational biology
Range-Renewal Structure in Continued Fractions

Jun Wu,Jian-Sheng Xie

DOI: https://doi.org/10.1017/etds.2015.91

2016-01-01

Abstract:Let $\unicode[STIX]{x1D714}=[a_{1},a_{2},\ldots ]$ be the infinite expansion of a continued fraction for an irrational number $\unicode[STIX]{x1D714}\in (0,1)$, and let $R_{n}(\unicode[STIX]{x1D714})$ (respectively, $R_{n,k}(\unicode[STIX]{x1D714})$, $R_{n,k+}(\unicode[STIX]{x1D714})$) be the number of distinct partial quotients, each of which appears at least once (respectively, exactly $k$ times, at least $k$ times) in the sequence $a_{1},\ldots ,a_{n}$. In this paper, it is proved that, for Lebesgue almost all $\unicode[STIX]{x1D714}\in (0,1)$ and all $k\geq 1$, $$\begin{eqnarray}\displaystyle \lim _{n\rightarrow \infty }\frac{R_{n}(\unicode[STIX]{x1D714})}{\sqrt{n}}=\sqrt{\frac{\unicode[STIX]{x1D70B}}{\log 2}},\quad \lim _{n\rightarrow \infty }\frac{R_{n,k}(\unicode[STIX]{x1D714})}{R_{n}(\unicode[STIX]{x1D714})}=\frac{C_{2k}^{k}}{(2k-1)\cdot 4^{k}},\quad \lim _{n\rightarrow \infty }\frac{R_{n,k}(\unicode[STIX]{x1D714})}{R_{n,k+}(\unicode[STIX]{x1D714})}=\frac{1}{2k}.\end{eqnarray}$$ The Hausdorff dimensions of certain level sets about $R_{n}$ are discussed.
Individual-level randomness in a nonatomic population

Edward J. Green

DOI: https://doi.org/10.48550/arXiv.1904.00849

2019-04-01

Abstract:This paper provides a construction of an uncountable family of i.i.d. random vectors, indexed by the points of a nonatomic measure space, such that (a) a sample is a measurable function from the index space, and (b) an idealization of the Glivenko-Cantelli theorem holds exactly with respect to the measure on that space. That is, samples possess a.s. the distribution from which they are drawn. Moreover, any subspace of the index space with positive measure inherits the same property. This homogeneity property is important for applications of the construction in economics.

Probability
A robust method for reliability updating with equality information using sequential adaptive importance sampling

Xiong Xiao,Zeyu Wang,Quanwang Li

DOI: https://doi.org/10.1016/j.cma.2023.116028

2023-03-08

Abstract:Reliability updating refers to a problem that integrates Bayesian updating technique with structural reliability analysis and cannot be directly solved by structural reliability methods (SRMs) when it involves equality information. The state-of-the-art approaches transform equality information into inequality information by introducing an auxiliary standard normal parameter. These methods, however, encounter the loss of computational efficiency due to the difficulty in finding the maximum of the likelihood function, the large coefficient of variation (COV) associated with the posterior failure probability and the inapplicability to dynamic updating problems where new information is constantly available. To overcome these limitations, this paper proposes an innovative method called RU-SAIS (reliability updating using sequential adaptive importance sampling), which combines elements of sequential importance sampling and K-means clustering to construct a series of important sampling densities (ISDs) using Gaussian mixture. The last ISD of the sequence is further adaptively modified through application of the cross entropy method. The performance of RU-SAIS is demonstrated by three examples. Results show that RU-SAIS achieves a more accurate and robust estimator of the posterior failure probability than the existing methods such as subset simulation.

Machine Learning

Range-Renewal Structure of I.I.D. Samplings

CLT, MDP and LDP for Range-Renewals of I.I.D.Samplings from an Infinite Discrete Distribution

Range-Renewal Speed and Entropy for I.I.D Models

On Increments of Sums of Independent Non-Identically Distributed Random-Variables

Range-Renewal Processes: SLLN, Power Law and Beyonds

Range-Renewal Structure of Transient Simple Random Walk

IID Sampling from Intractable Distributions

Extension of Donsker's Invariance Principle with Incomplete Partial-Sum Process

Near-linear time samplers for matroid independent sets with applications

Independent Range Sampling on Interval Data (Longer Version)

Achieving Efficiency in Black Box Simulation of Distribution Tails with Self-structuring Importance Samplers

SINGLE-RECORD RANDOM INCREMENTAL DYNAMIC ANALYSIS

A LIL for Independent Non-Identically Distributed Random Variables in Banach Space and Its Applications

Risk Bounds for Infinitely Divisible Distribution

Optimal scaling of the independence sampler: Theory and practice

Range-Renewal Processes: SLLNs and Power Laws

Subset Sampling and Its Extensions

A renewal theory approach to IBD sharing

Range-Renewal Structure in Continued Fractions

Individual-level randomness in a nonatomic population

A robust method for reliability updating with equality information using sequential adaptive importance sampling