Abstract:ABSTRACT Understanding the ecological impacts of viruses on natural and engineered ecosystems relies on the accurate identification of viral sequences from community sequencing data. To maximize viral recovery from metagenomes, researchers frequently combine viral identification tools. However, the effectiveness of this strategy is unknown. Here, we benchmarked combinations of six widely used informatics tools for viral identification and analysis (VirSorter, VirSorter2, VIBRANT, DeepVirFinder, CheckV, and Kaiju), called “rulesets.” Rulesets were tested against mock metagenomes composed of taxonomically diverse sequence types and diverse aquatic metagenomes to assess the effects of the degree of viral enrichment and habitat on tool performance. We found that six rulesets achieved equivalent accuracy [Matthews Correlation Coefficient (MCC) = 0.77, P adj ≥ 0.05]. Each contained VirSorter2, and five used our “tuning removal” rule designed to remove non-viral contamination. While DeepVirFinder, VIBRANT, and VirSorter were each found once in these high-accuracy rulesets, they were not found in combination with each other: combining tools does not lead to optimal performance. Our validation suggests that the MCC plateau at 0.77 is partly caused by inaccurate labeling within reference sequence databases. In aquatic metagenomes, our highest MCC ruleset identified more viral sequences in virus-enriched (44%–46%) than in cellular metagenomes (7%–19%). While improved algorithms may lead to more accurate viral identification tools, this should be done in tandem with careful curation of sequence databases. We recommend using the VirSorter2 ruleset and our empirically derived tuning removal rule. Our analysis provides insight into methods for in silico viral identification and will enable more robust viral identification from metagenomic data sets. IMPORTANCE The identification of viruses from environmental metagenomes using informatics tools has offered critical insights in microbial ecology. However, it remains difficult for researchers to know which tools optimize viral recovery for their specific study. In an attempt to recover more viruses, studies are increasingly combining the outputs from multiple tools without validating this approach. After benchmarking combinations of six viral identification tools against mock metagenomes and environmental samples, we found that these tools should only be combined cautiously. Two to four tool combinations maximized viral recovery and minimized non-viral contamination compared with either the single-tool or the five- to six-tool ones. By providing a rigorous overview of the behavior of in silico viral identification strategies and a pipeline to replicate our process, our findings guide the use of existing viral identification tools and offer a blueprint for feature engineering of new tools that will lead to higher-confidence viral discovery in microbiome studies.

Benchmarking bioinformatic virus identification tools using real-world metagenomic data across biomes

Benchmarking informatics approaches for virus discovery: caution is needed when combining in silico identification methods

Gauge your phage: benchmarking of bacteriophage identification tools in metagenomic sequencing data

Bioinformatic Approaches for Comparative Analysis of Viruses

Benchmarking Metagenomics Tools for Taxonomic Classification

VirBot: an RNA Viral Contig Detector for Metagenomic Data.

Multicenter benchmarking of short and long read wet lab protocols for clinical viral metagenomics

Novel Virus Identification through Metagenomics: A Systematic Review

Identifying viruses from metagenomic data by deep learning

Complementary Insights into Gut Viral Genomes: a Comparative Benchmark of Short- and Long-Read Metagenomes Using Diverse Assemblers and Binners

Microbial and Viral Ecology Analysis for Metagenomic Data

VIGA: a one-stop tool for eukaryotic virus identification and genome assembly from next-generation-sequencing data

Bioinformatic Tools for NGS-Based Metagenomics to Improve the Clinical Diagnosis of Emerging, Re-Emerging and New Viruses

Evaluating metagenomics and targeted approaches for diagnosis and surveillance of viruses

VirID: Beyond Virus Discovery - An Integrated Platform for Comprehensive RNA Virus Characterization

Comparative evaluation of bioinformatic tools for virus-host prediction and their application to a highly diverse community in the Cuatro Ciénegas Basin, Mexico

The Long and Short of It: Benchmarking Viromics Using Illumina, Nanopore and PacBio Sequencing Technologies

Comprehensive comparative analysis of methods and software for identifying viral integrations.

A Benchmark of Genetic Variant Calling Pipelines Using Metagenomic Short-Read Sequencing

VirFinder: a novel k-mer based tool for identifying viral sequences from assembled metagenomic data