Abstract:Background and objective Short-read sequencing is becoming the standard of practice for the study of structural variants associated with disease. However, with the growth of sequence data largely surpassing reasonable storage capability, the biomedical community is challenged with the management, transfer, archiving, and storage of sequence data.Methods We developed Hierarchical mUlti-reference Genome cOmpression (HUGO), a novel compression algorithm for aligned reads in the sorted Sequence Alignment/Map (SAM) format. We first aligned short reads against a reference genome and stored exactly mapped reads for compression. For the inexact mapped or unmapped reads, we realigned them against different reference genomes using an adaptive scheme by gradually shortening the read length. Regarding the base quality value, we offer lossy and lossless compression mechanisms. The lossy compression mechanism for the base quality values uses k-means clustering, where a user can adjust the balance between decompression quality and compression rate. The lossless compression can be produced by setting k (the number of clusters) to the number of different quality values.Results The proposed method produced a compression ratio in the range 0.5-0.65, which corresponds to 35-50% storage savings based on experimental datasets. The proposed approach achieved 15% more storage savings over CRAM and comparable compression ratio with Samcomp (CRAM and Samcomp are two of the state-of-the-art genome compression algorithms). The software is freely available at https://sourceforge.net/projects/hierachicaldnac/with a General Public License (GPL) license.Limitation Our method requires having different reference genomes and prolongs the execution time for additional alignments.Conclusions The proposed multi-reference-based compression algorithm for aligned reads outperforms existing single-reference based algorithms.

Centrifuger: lossless compression of microbial genomes for efficient and accurate metagenomic sequence classification

Fidbac: A Platform for Fast Bacterial Genome Identification and Typing

MCUIUC -- A New Framework for Metagenomic Read Compression

An experimental sorting method for improving metagenomic data encoding

The MetaGens algorithm for metagenomic database lossy compression and subject alignment

Efficient and Robust Search of Microbial Genomes via Phylogenetic Compression

AMGC: Adaptive match-based genomic compression algorithm

Real-time Taxonomic Characterization of Long-read Mixed-species Sequencing Samples in Sorted Motif Distance Space:

Identification and quantitation of clinically relevant microbes in patient samples: Comparison of three k-mer based classifiers for speed, accuracy, and sensitivity

GeneSqueeze: A Novel Lossless, Reference-Free Compression Algorithm for FASTQ/A Files

GSC: efficient lossless compression of VCF files with fast query

FastqZip: An Improved Reference-Based Genome Sequence Lossy Compression Framework

Compressing genomic sequence fragments using SlimGene

HUGO: Hierarchical Multi-Reference Genome Compression for Aligned Reads.

Lossy Compressor preserving variant calling through Extended BWT

A read-filtering algorithm for high-throughput marker-gene studies that greatly improves OTU accuracy

Distilled Single Cell Genome Sequencing and De Novo Assembly for Sparse Microbial Communities

CMIC: an efficient quality score compressor with random access functionality

Handling the data management needs of high-throughput sequencing data: SpeedGene, a compression algorithm for the efficient storage of genetic data

Despi: Efficient Classification of Metagenomics Reads with Lightweight De Bruijn Graph-Based Reference Indexing

A Pipeline for Constructing Reference Genomes for Large Cohort-Specific Metagenome Compression