Abstract:BackgroundMicrobiome studies commonly use 16S rRNA gene amplicon sequencing to characterize microbial communities. Errors introduced at multiple steps in this process can affect the interpretation of the data. Here we evaluate the accuracy of operational taxonomic unit (OTU) generation, taxonomic classification, alpha- and beta-diversity measures for different settings in QIIME, MOTHUR and a pplacer-based classification pipeline, using a novel software package: DECARD.ResultsIn-silico we generated 100 synthetic bacterial communities approximating human stool microbiomes to be used as a gold-standard for evaluating the colligative performance of microbiome analysis software. Our synthetic data closely matched the composition and complexity of actual healthy human stool microbiomes. Genus-level taxonomic classification was correctly done for only 50.4–74.8% of the source organisms. Miscall rates varied from 11.9 to 23.5%. Species-level classification was less successful, (6.9–18.9% correct); miscall rates were comparable to those of genus-level targets (12.5–26.2%). The degree of miscall varied by clade of organism, pipeline and specific settings used. OTU generation accuracy varied by strategy (closed, de novo or subsampling), reference database, algorithm and software implementation. Shannon diversity estimation accuracy correlated generally with OTU-generation accuracy. Beta-diversity estimates with Double Principle Coordinate Analysis (DPCoA) were more robust against errors introduced in processing than Weighted UniFrac. The settings suggested in the tutorials were among the worst performing in all outcomes tested.ConclusionsEven when using the same classification pipeline, the specific OTU-generation strategy, reference database and downstream analysis methods selection can have a dramatic effect on the accuracy of taxonomic classification, and alpha- and beta-diversity estimation. Even minor changes in settings adversely affected the accuracy of the results, bringing them far from the best-observed result. Thus, specific details of how a pipeline is used (including OTU generation strategy, reference sets, clustering algorithm and specific software implementation) should be specified in the methods section of all microbiome studies. Researchers should evaluate their chosen pipeline and settings to confirm it can adequately answer the research question rather than assuming the tutorial or standard-operating-procedure settings will be adequate or optimal.

Updating the 97% identity threshold for 16S ribosomal RNA OTUs

Subsampled Open-Reference Clustering Creates Consistent, Comprehensive OTU Definitions and Scales to Billions of Sequences

Two-Stage Clustering (Tsc): A Pipeline For Selecting Operational Taxonomic Units For The High-Throughput Sequencing Of Pcr Amplicons

Interpreting 16S metagenomic data without clustering to achieve sub-OTU resolution

Uniting the classification of cultured and uncultured bacteria and archaea using 16S rRNA gene sequences

Comparison of Methods for Picking the Operational Taxonomic Units From Amplicon Sequences

ASV vs OTUs clustering: Effects on alpha, beta, and gamma diversities in microbiome metabarcoding studies

Clustering 16S Rrna for OTU Prediction: a Method of Unsupervised Bayesian Clustering.

Towards a Taxonomical Consensus: Diversity and Richness Inference from Large Scale rRNA gene Analysis

Defining Loci in Restriction-Based Reduced Representation Genomic Data from Nonmodel Species: Sources of Bias and Diagnostics for Optimal Clustering

Amplicon Sequence Variants Artificially Split Bacterial Genomes into Separate Clusters

Evaluation of 16S rRNA gene sequencing for species and strain-level microbiome analysis

Unassigning bacterial species for microbiome studies

Metagenomic profiling pipelines improve taxonomic classification for 16S amplicon sequencing data

Using Mothur to Determine Bacterial Community Composition and Structure in 16S Ribosomal RNA Datasets

Evaluating the accuracy of amplicon-based microbiome computational pipelines on simulated human gut microbial communities

Confidence Intervals of Similarity Values Determined for Cloned Ssu Rrna Genes from Environmental Samples

Benchmarking of 16S rRNA gene databases using known strain sequences

Impact of training sets on classification of high-throughput bacterial 16s rRNA gene surveys

To compare the performance of prokaryotic taxonomy classifiers using curated 16S full-length rRNA sequences

16S rRNA phylogeny and clustering is not a reliable proxy for genome-based taxonomy in Streptomyces