Abstract:Read mapping is a fundamental, yet computationally-expensive step in many genomics applications. It is used to identify potential matches and differences between fragments (called reads) of a sequenced genome and an already known genome (called a reference genome). To address the computational challenges in genome analysis, many prior works propose various approaches such as filters that select the reads that must undergo expensive computation, efficient heuristics, and hardware acceleration. While effective at reducing the computation overhead, all such approaches still require the costly movement of a large amount of data from storage to the rest of the system, which can significantly lower the end-to-end performance of read mapping in conventional and emerging genomics systems. We propose GenStore, the first in-storage processing system designed for genome sequence analysis that greatly reduces both data movement and computational overheads of genome sequence analysis by exploiting low-cost and accurate in-storage filters. GenStore leverages hardware/software co-design to address the challenges of in-storage processing, supporting reads with 1) different read lengths and error rates, and 2) different degrees of genetic variation. Through rigorous analysis of read mapping processes, we meticulously design low-cost hardware accelerators and data/computation flows inside a NAND flash-based SSD. Our evaluation using a wide range of real genomic datasets shows that GenStore, when implemented in three modern SSDs, significantly improves the read mapping performance of state-of-the-art software (hardware) baselines by 2.07-6.05$\times$ (1.52-3.32$\times$) for read sets with high similarity to the reference genome and 1.45-33.63$\times$ (2.70-19.2$\times$) for read sets with low similarity to the reference genome.

Handling the data management needs of high-throughput sequencing data: SpeedGene, a compression algorithm for the efficient storage of genetic data

GeneZip: A software package for storage-efficient processing of genotype data

Gene Sequence Alignment on a Public Computing Platform

GSC: efficient lossless compression of VCF files with fast query

High efficiency referential genome compression algorithm.

Efficient Direct Search On Compressed Genomic Data

Compressing genomic sequence fragments using SlimGene

A Novel Compression Tool For Efficient Storage Of Genome Resequencing Data

GeneSqueeze: A Novel Lossless, Reference-Free Compression Algorithm for FASTQ/A Files

Heterogeneous Genome Compression on Mobile Devices

GPU-accelerated adaptive compression framework for genomics data

FastqZip: An Improved Reference-Based Genome Sequence Lossy Compression Framework

Compression of Structured High-Throughput Sequencing Data

CARGO: effective format-free compressed storage of genomic information

Efficient storage and regression computation for population-scale genome sequencing studies

GVC: efficient random access compression for gene sequence variations

A Novel Compression Algorithm for High-Throughput DNA Sequence Based on Huffman Coding Method

AMGC: Adaptive match-based genomic compression algorithm

Compression with Unified and Accessible Byte Blocks to Enhance Management and Analyses of UKBB-scale Genotypes

GenStore: A High-Performance and Energy-Efficient In-Storage Computing System for Genome Sequence Analysis

PLDSRC: A Multi-threaded Compressor/Decompressor for Massive DNA Sequencing Data