Abstract:Motivation: Pileup analysis is a building block of many bioinformatics pipelines, including variant calling and genotyping. This step tends to become a bottleneck of the entire assay since the straightforward pileup implementations involve processing of all base calls from all alignments sequentially. On the other hand, a distributed version of the algorithm faces the intrinsic challenge of splitting reads-oriented file formats into self-contained partitions to avoid costly data exchange between computational nodes. Results: Here, we present a scalable, distributed and efficient implementation of a pileup algorithm that is suitable for deploying in cloud computing environments. In particular, we implemented: (i) our custom data-partitioning algorithm optimized to work with the alignment reads, (ii) a novel and unique approach to process alignment events from sequencing reads using the MD tags, (iii) the source code micro-optimizations for recurrent operations, and (iv) a modular structure of the algorithm. We have proven that our novel approach consistently and significantly outperforms other state-of-the-art distributed tools in terms of execution time (up to 6.5× faster) and memory usage (up to 2× less), resulting in a substantial cloud cost reduction. SeQuiLa is a cloud-native solution that can be easily deployed using any managed Kubernetes and Hadoop services available in public clouds, like Microsoft Azure Cloud, Google Cloud Platform, or Amazon Web Services. Together with the already implemented distributed range join and coverage calculations, our package provides end-users with a unified SQL interface for convenient analyses of population-scale genomic data in an interactive way. Availability and implementation: https://biodatageeks.github.io/sequila/.

Cloud Based Short Read Mapping Service

SCAN: A Smart Application Platform for Empowering Parallelizations of Big Genomic Data Analysis in Clouds

Bwasw-Cloud: Efficient sequence alignment algorithm for two big data with MapReduce

Gene Sequence Alignment on a Public Computing Platform

Genome sequencing using mapreduce on FPGA with multiple hardware accelerators (abstract only).

Interoperable RNA-Seq analysis in the cloud

Heterogeneous Cloud Framework for Big Data Genome Sequencing.

miCloud: A Plug-n-Play, Extensible, On-Premises Bioinformatics Cloud for Seamless Execution of Complex Next-Generation Sequencing Data Analysis Pipelines

Distributed Short Read Mapping System

Short read mapping for exome sequencing.

An efficient Burrows-Wheeler transform-based aligner for short read mapping

When cloud computing meets bioinformatics: a review.

CloudNMF: a MapReduce Implementation of Nonnegative Matrix Factorization for Large-Scale Biological Datasets.

Privacy-Preserving Read Mapping Using Locality Sensitive Hashing and Secure Kmer Voting

Parallelization of BLAST with MapReduce for Long Sequence Alignment.

MUSIC: A Hybrid Computing Environment for Burrows-Wheeler Alignment for Massive Amount of Short Read Sequence Data

Cloud Computing Service Framework for Bioinformatics Tools

A Re-Sequencing Tool For High Mismatch-Tolerant Short Read Alignment Based On Burrows-Wheeler Transform

TreQ-CG: Clustering Accelerates High-Throughput Sequencing Read Mapping

ReadsMap: a new tool for high precision mapping of DNAseq and RNAseq read sequences

Cloud-native distributed genomic pileup operations