Abstract:ABSTRACT Understanding the ecological impacts of viruses on natural and engineered ecosystems relies on the accurate identification of viral sequences from community sequencing data. To maximize viral recovery from metagenomes, researchers frequently combine viral identification tools. However, the effectiveness of this strategy is unknown. Here, we benchmarked combinations of six widely used informatics tools for viral identification and analysis (VirSorter, VirSorter2, VIBRANT, DeepVirFinder, CheckV, and Kaiju), called “rulesets.” Rulesets were tested against mock metagenomes composed of taxonomically diverse sequence types and diverse aquatic metagenomes to assess the effects of the degree of viral enrichment and habitat on tool performance. We found that six rulesets achieved equivalent accuracy [Matthews Correlation Coefficient (MCC) = 0.77, P adj ≥ 0.05]. Each contained VirSorter2, and five used our “tuning removal” rule designed to remove non-viral contamination. While DeepVirFinder, VIBRANT, and VirSorter were each found once in these high-accuracy rulesets, they were not found in combination with each other: combining tools does not lead to optimal performance. Our validation suggests that the MCC plateau at 0.77 is partly caused by inaccurate labeling within reference sequence databases. In aquatic metagenomes, our highest MCC ruleset identified more viral sequences in virus-enriched (44%–46%) than in cellular metagenomes (7%–19%). While improved algorithms may lead to more accurate viral identification tools, this should be done in tandem with careful curation of sequence databases. We recommend using the VirSorter2 ruleset and our empirically derived tuning removal rule. Our analysis provides insight into methods for in silico viral identification and will enable more robust viral identification from metagenomic data sets. IMPORTANCE The identification of viruses from environmental metagenomes using informatics tools has offered critical insights in microbial ecology. However, it remains difficult for researchers to know which tools optimize viral recovery for their specific study. In an attempt to recover more viruses, studies are increasingly combining the outputs from multiple tools without validating this approach. After benchmarking combinations of six viral identification tools against mock metagenomes and environmental samples, we found that these tools should only be combined cautiously. Two to four tool combinations maximized viral recovery and minimized non-viral contamination compared with either the single-tool or the five- to six-tool ones. By providing a rigorous overview of the behavior of in silico viral identification strategies and a pipeline to replicate our process, our findings guide the use of existing viral identification tools and offer a blueprint for feature engineering of new tools that will lead to higher-confidence viral discovery in microbiome studies.

Gauge your phage: benchmarking of bacteriophage identification tools in metagenomic sequencing data

Benchmarking informatics approaches for virus discovery: caution is needed when combining in silico identification methods

Benchmarking bioinformatic virus identification tools using real-world metagenomic data across biomes

PHISDetector: A Tool to Detect Diverse in Silico Phage-host Interaction Signals for Virome Studies

Jaeger: an accurate and fast deep-learning tool to detect bacteriophage sequences

VirFinder: a novel k-mer based tool for identifying viral sequences from assembled metagenomic data

The Long and Short of It: Benchmarking Viromics Using Illumina, Nanopore and PacBio Sequencing Technologies

DeePhage: Distinguishing Virulent and Temperate Phage-Derived Sequences in Metavirome Data with a Deep Learning Approach

Biological and bioinformatic tools for the discovery of unknown phage–host combinations

Complementary Insights into Gut Viral Genomes: a Comparative Benchmark of Short- and Long-Read Metagenomes Using Diverse Assemblers and Binners

A Bioinformatic Ecosystem for Bacteriophage Genomics: PhaMMSeqs, Phamerator, pdm_utils, PhagesDB, DEPhT, and PhamClust

Phage family classification under Caudoviricetes: a review of current tools using the latest ICTV classification framework

Phage-Host Prediction Using a Computational Tool Coupled with 16S rRNA Gene Amplicon Sequencing

HoPhage: an Ab Initio Tool for Identifying Hosts of Phage Fragments from Metaviromes.

Microseek: A Protein-Based Metagenomic Pipeline for Virus Diagnostic and Discovery

Identify phage hosts from metaviromic short reads based on deep learning and Markov chain model

Improving gut virome comparisons using predicted phage host information

A Benchmark of Genetic Variant Calling Pipelines Using Metagenomic Short-Read Sequencing

PhageScanner: a reconfigurable machine learning framework for bacteriophage genomic and metagenomic feature annotation

Metavir 2: new tools for viral metagenome comparison and assembled virome analysis