Abstract:MOTIVATION: The Burrows-Wheeler transform (BWT) is the foundation of many algorithms for compression and indexing of text data, but the cost of computing the BWT of very large string collections has prevented these techniques from being widely applied to the large sets of sequences often encountered as the outcome of DNA sequencing experiments. In previous work, we presented a novel algorithm that allows the BWT of human genome scale data to be computed on very moderate hardware, thus enabling us to investigate the BWT as a tool for the compression of such datasets.RESULTS: We first used simulated reads to explore the relationship between the level of compression and the error rate, the length of the reads and the level of sampling of the underlying genome and compare choices of second-stage compression algorithm. We demonstrate that compression may be greatly improved by a particular reordering of the sequences in the collection and give a novel 'implicit sorting' strategy that enables these benefits to be realized without the overhead of sorting the reads. With these techniques, a 45× coverage of real human genome sequence data compresses losslessly to under 0.5 bits per base, allowing the 135.3 Gb of sequence to fit into only 8.2 GB of space (trimming a small proportion of low-quality bases from the reads improves the compression still further). This is >4 times smaller than the size achieved by a standard BWT-based compressor (bzip2) on the untrimmed reads, but an important further advantage of our approach is that it facilitates the building of compressed full text indexes such as the FM-index on large-scale DNA sequence collections.AVAILABILITY: Code to construct the BWT and SAP-array on large genomic datasets is part of the BEETL library, available as a github repository at https://github.com/BEETL/BEETL.

Ultrafast and memory-efficient alignment of short DNA sequences to the human genome

Fast and accurate short read alignment with hybrid hash-tree data structure

Fast and accurate long-read alignment with Burrows–Wheeler transform

Gene Sequence Alignment on a Public Computing Platform

An efficient Burrows-Wheeler transform-based aligner for short read mapping

CGAP-align: a high performance DNA short read alignment tool.

Aligning sequence reads, clone sequences and assembly contigs with BWA-MEM

A Re-Sequencing Tool For High Mismatch-Tolerant Short Read Alignment Based On Burrows-Wheeler Transform

MUSIC: A Hybrid Computing Environment for Burrows-Wheeler Alignment for Massive Amount of Short Read Sequence Data

Minimap2: pairwise alignment for nucleotide sequences

SOAP2: an improved ultrafast tool for short read alignment

BOAT: Basic Oligonucleotide Alignment Tool

Fast and Accurate Read Alignment for Resequencing.

Pblat: a Multithread Blat Algorithm Speeding Up Aligning Sequences to Genomes.

Sensitive Long-Indel-Aware Alignment of Sequencing Reads

FPGA Acceleration of Short Read Alignment

A fast read alignment method based on seed-and-vote for next generation sequencing

Large-scale compression of genomic sequence databases with the Burrows-Wheeler transform

Accelerating whole-genome alignment in the age of complete genome assemblies

A DNA Sequence Alignment Tool Based on BWA and Data Mining

Efficient Memory Layout for Pre-Alignment Filtering of Long DNA Reads Using Racetrack Memory