Abstract:Quantifying and assessing changes in biological diversity are central aspects of many ecological studies, yet accurate methods of estimating biological diversity from sampling data have been elusive. Hill numbers, or the effective number of species, are increasingly used to characterize the taxonomic, phylogenetic, or functional diversity of an assemblage. However, empirical estimates of Hill numbers, including species richness, tend to be an increasing function of sampling effort and, thus, tend to increase with sample completeness. Integrated curves based on sampling theory that smoothly link rarefaction (interpolation) and prediction (extrapolation) standardize samples on the basis of sample size or sample completeness and facilitate the comparison of biodiversity data. Here we extended previous rarefaction and extrapolation models for species richness (Hill number qD, where q = 0) to measures of taxon diversity incorporating relative abundance (i.e., for any Hill number qD, q > 0) and present a unified approach for both individual‐based (abundance) data and sample‐based (incidence) data. Using this unified sampling framework, we derive both theoretical formulas and analytic estimators for seamless rarefaction and extrapolation based on Hill numbers. Detailed examples are provided for the first three Hill numbers: q = 0 (species richness), q = 1 (the exponential of Shannon's entropy index), and q = 2 (the inverse of Simpson's concentration index). We developed a bootstrap method for constructing confidence intervals around Hill numbers, facilitating the comparison of multiple assemblages of both rarefied and extrapolated samples. The proposed estimators are accurate for both rarefaction and short‐range extrapolation. For long‐range extrapolation, the performance of the estimators depends on both the value of q and on the extrapolation range. We tested our methods on simulated data generated from species abundance models and on data from large species inventories. We also illustrate the formulas and estimators using empirical data sets from biodiversity surveys of temperate forest spiders and tropical ants.

Estimating diversity via frequency ratios

Rarefaction, Alpha Diversity, and Statistics

Species richness estimation with high diversity but spurious singletons

How to partition diversity

Nonparametric Estimation of Repeated Densities with Heterogeneous Sample Sizes

Robust estimation of microbial diversity in theory and in practice

The measurement of diversity in different types of biological collections

Estimating Diversity Through Time using Molecular Phylogenies: Old and Species-Poor Frog Families are the Remnants of a Diverse Past

Functional Alpha Diversity: A New Distance-Based Measurement Calculations Technique That Accounts for Intraspecific Variability

Rarefaction and extrapolation with Hill numbers: a framework for sampling and estimation in species diversity studies

Population size estimation based upon ratios of recapture probabilities

Analysis of Frequency Count Data Using the Negative Binomial Distribution

Estimating the Density Ratio between Distributions with High Discrepancy using Multinomial Logistic Regression

Estimating Unknown Population Sizes Using the Hypergeometric Distribution

Estimation of mean using under-reported and overdispersed count data

Extrapolation of Urn Models via Poissonization: Accurate Measurements of the Microbial Unknown

Isolating selective from non-selective forces using site frequency ratios

rarestR: An R package using rarefaction metrics to estimate α-diversity (species richness) and β-diversity (species shared) for incomplete samples

Kernel density estimation of allele frequency including undetected alleles

A simple Bayesian model to estimate proportions and ratios from count data with a hierarchical error structure with an application to droplet digital PCR experiments

Modeling microbial abundances and dysbiosis with beta-binomial regression