Abstract:The availability of high-throughput parallel methods for sequencing microbial communities is increasing our knowledge of the microbial world at an unprecedented rate. Though most attention has focused on determining lower-bounds on the alpha-diversity i.e. the total number of different species present in the environment, tight bounds on this quantity may be highly uncertain because a small fraction of the environment could be composed of a vast number of different species. To better assess what remains unknown, we propose instead to predict the fraction of the environment that belongs to unsampled classes. Modeling samples as draws with replacement of colored balls from an urn with an unknown composition, and under the sole assumption that there are still undiscovered species, we show that conditionally unbiased predictors and exact prediction intervals (of constant length in logarithmic scale) are possible for the fraction of the environment that belongs to unsampled classes. Our predictions are based on a Poissonization argument, which we have implemented in what we call the Embedding algorithm. In fixed i.e. non-randomized sample sizes, the algorithm leads to very accurate predictions on a sub-sample of the original sample. We quantify the effect of fixed sample sizes on our prediction intervals and test our methods and others found in the literature against simulated environments, which we devise taking into account datasets from a human-gut and -hand microbiota. Our methodology applies to any dataset that can be conceptualized as a sample with replacement from an urn. In particular, it could be applied, for example, to quantify the proportion of all the unseen solutions to a binding site problem in a random RNA pool, or to reassess the surveillance of a certain terrorist group, predicting the conditional probability that it deploys a new tactic in a next attack.

Estimation of Distribution Overlap of Urn Models

Extrapolation of Urn Models via Poissonization: Accurate Measurements of the Microbial Unknown

Asymptotic Theorems Of Sequential Estimation-Adjusted Urn Models

Gaussian Approximation Theorems for Urn Models and Their Applications

Asymptotic Properties of Multicolor Randomly Reinforced Pólya Urns

A Gaussian Process Approximation for Two-Color Randomly Reinforced Urns

Convergence of randomized urn models with irreducible and reducible replacement policy

A generalized urn model in clinical trials

The Gaussian Approximation for Generalized Friedman's Urn Model with Heterogeneous and Unbalanced Updating

A New Method for Computing the Expected Hitting Time Between Arbitrary Different Configurations of the Multiple-Urn Ehrenfest Model

Estimation of Distribution Algorithm Using Variety of Information

Large deviations of a long-time average in the Ehrenfest Urn Model

Immigrated urn models - asymptotic properties and applications

How to decide whether small samples comply with an equidistribution

Analysis of Large Urn Models with Local Mean-Field Interactions

Combinatorics of a dissimilarity measure for pairs of draws from discrete probability vectors on finite sets of objects

Cutoff in the Bernoulli-Laplace Model With Unequal Colors and Urn Sizes

Urns with Multiple Drawings and Graph-Based Interaction

A mixture distribution for modelling bivariate ordinal data

Models Based on Exponential Interarrival Times for Single-Unusual-Event Count Data