Abstract:The availability of high-throughput parallel methods for sequencing microbial communities is increasing our knowledge of the microbial world at an unprecedented rate. Though most attention has focused on determining lower-bounds on the alpha-diversity i.e. the total number of different species present in the environment, tight bounds on this quantity may be highly uncertain because a small fraction of the environment could be composed of a vast number of different species. To better assess what remains unknown, we propose instead to predict the fraction of the environment that belongs to unsampled classes. Modeling samples as draws with replacement of colored balls from an urn with an unknown composition, and under the sole assumption that there are still undiscovered species, we show that conditionally unbiased predictors and exact prediction intervals (of constant length in logarithmic scale) are possible for the fraction of the environment that belongs to unsampled classes. Our predictions are based on a Poissonization argument, which we have implemented in what we call the Embedding algorithm. In fixed i.e. non-randomized sample sizes, the algorithm leads to very accurate predictions on a sub-sample of the original sample. We quantify the effect of fixed sample sizes on our prediction intervals and test our methods and others found in the literature against simulated environments, which we devise taking into account datasets from a human-gut and -hand microbiota. Our methodology applies to any dataset that can be conceptualized as a sample with replacement from an urn. In particular, it could be applied, for example, to quantify the proportion of all the unseen solutions to a binding site problem in a random RNA pool, or to reassess the surveillance of a certain terrorist group, predicting the conditional probability that it deploys a new tactic in a next attack.

Confidence intervals for nonhomogeneous branching processes and polymerase chain reactions

An Empirical Bayes Approach for Constructing the Confidence Intervals of Clonality and Entropy

Statistical Inference for Quantitative Polymerase Chain Reaction Using a Hidden Markov Model: A Bayesian Approach

Alpha-stable branching and beta-Frequency processes, beyond the IID assumption

On estimation of the convergence rate to invariant measures in markov branching processes with possibly infinite variance and allowing immigration

Approximations of the allelic frequency spectrum in general supercritical branching populations

Controlling Uncertainty of Empirical First-Passage Times in the Small-Sample Regime

Estimation of the lifetime distribution from fluctuations in Bellman-Harris processes

Limit theorems for time averages of continuous-state branching processes with immigration

Strongly Convergent Homogeneous Approximations to Inhomogeneous Markov Jump Processes and Applications

Binary branching processes with Moran type interactions

Sequential mutations in exponentially growing populations

Stable continuous branching processes with immigration and Beta-Fleming-Viot processes with immigration

Formal properties of the probability of fixation: identities, inequalities and approximations

Mixing it up: A general framework for Markovian statistics

A central limit theorem concerning uncertainty in estimates of individual admixture

Weak convergence of the scaled jump chain and number of mutations of the Kingman coalescent

Lévy processes with marked jumps II : Application to a population model with mutations at birth

Extrapolation of Urn Models via Poissonization: Accurate Measurements of the Microbial Unknown

Exact confidence intervals for population growth rate, longevity and generation time