Abstract:Suffix Array (SA) is a cardinal data structure in many pattern matching applications, including data compression, plagiarism detection and sequence alignment. However, as the volumes of data increase abruptly, the construction of SA is not amenable to the current large-scale data processing frameworks anymore due to its intrinsic proliferation of suffixes during the construction. That is, ameliorating the performance by just adding the resources to the frameworks becomes less cost- effective, even having the severe diminishing returns. At issue now is whether we can permit SA construction to be more scalable and efficient for the everlasting accretion of data by creating a radical shift in perspective. Regarding TeraSort [1] as our baseline, we first demonstrate the fragile scalability of TeraSort and investigate what causes it through the experiments on the sequence alignment of a grouper (i.e., the SA construc- tion used in bioinformatics). As such, we propose a scheme that amalgamates the distributed key-value store system into MapReduce to leverage the in-memory queries about suffixes. Rather than handling the communication of suffixes, MapReduce is in charge of the communication of their indexes, which means better capacity for more data. It significantly abates the required disk space for constructing SA and better utilizes the memory, which in turn improves the scalability radically. We also examine the efficiency of our scheme in terms of memory and show it outperforms TeraSort. At last, our scheme can complete the pair- end sequencing and alignment with two input files without any degradation on scalability, and can accommodate the suffixes of nearly 6.7 TB in a small cluster composed of 16 nodes and Gigabit Ethernet without any compression.

Using GPU to Accelerate Suffix Array Construction

Parallel Lexicographic Names Construction with CUDA

Fast, parallel, and cache-friendly suffix array construction

Fast and Lightweight Distributed Suffix Array Construction -- First Results

DGST: Efficient and Scalable Suffix Tree Construction on Distributed Data-Parallel Platforms.

GPU Accelerated Biological Sequence Alignment

Accelerating String Matching Using Multi-Threaded Algorithm on GPU

CUDAMPF++: A Proactive Resource Exhaustion Scheme for Accelerating Homologous Sequence Search on CUDA-enabled GPU

Gac: High Performance AC Algorithm Based on GPU

An efficient parallel algorithm for longest common subsequence problem on GPUs

Parallelizing Dynamic Time Warping Algorithm Using Prefix Computations on GPU

Generic Non-Recursive Suffix Array Construction

A new parallel method of smith-waterman algorithm on a heterogeneous platform

Accelerating Biological Sequence Alignment Algorithm on GPU with CUDA

A fast divide-and-conquer algorithm for indexing human genome sequences

Parallel Identifying (l, d)-Motifs in Biosequences Using CPU and GPU Computing.

The optimization of parallel Smith-Waterman sequence alignment using on-chip memory of GPGPU

gpuPairHMM: High-speed Pair-HMM Forward Algorithm for DNA Variant Calling on GPUs

Systolic Computing on GPUs for Productive Performance

Scalable and Efficient Construction of Suffix Array with MapReduce and In-Memory Data Store System

Optimization of Linked List Prefix Computations on Multithreaded GPUs Using CUDA