Abstract:ABSTRACT Rapid technological advancements in sequencing technologies allow producing cost effective and high volume sequencing data. Processing this data for real-time clinical diagnosis is potentially time-consuming if done on a single computing node. This work presents a complete variant calling workflow, implemented using the Message Passing Interface (MPI) to leverage the benefits of high bandwidth interconnects. This solution (GenMPI) is portable and flexible, meaning it can be deployed to any private or public cluster/cloud infrastructure. Any alignment or variant calling application can be used with minimal adaptation. To achieve high performance, compressed input data can be streamed in parallel to alignment applications while uncompressed data can use internal file seek functionality to eliminate the bottleneck of streaming input data from a single node. Alignment output can be directly stored in multiple chromosome-specific SAM files or a single SAM file. After alignment, a distributed queue using MPI RMA (Remote Memory Access) atomic operations is created for sorting, indexing, marking of duplicates (if necessary) and variant calling applications. We ensure the accuracy of variants as compared to the original single node methods. We also show that for 300x coverage data, alignment scales almost linearly up to 64 nodes (8192 CPU cores). Overall, this work outperforms existing big data based workflows by a factor of two and is almost 20% faster than other MPI-based implementations for alignment without any extra memory overheads. Sorting, indexing, duplicate removal and variant calling is also scalable up to 8 nodes cluster. For pair-end short-reads (Illumina) data, we integrated the BWA-MEM aligner and three variant callers (GATK HaplotypeCaller, DeepVariant and Octopus), while for long-reads data, we integrated the Minimap2 aligner and three different variant callers (DeepVariant, DeepVariant with WhatsHap for phasing (PacBio) and Clair3 (ONT)). All codes and scripts are available at: https://github.com/abs-tudelft/gen-mpi

Acceleration of short and long DNA read mapping without loss of accuracy using suffix array

CGAP-align: a high performance DNA short read alignment tool.

Sensitive Long-Indel-Aware Alignment of Sequencing Reads

GMAP and GSNAP for Genomic Sequence Alignment: Enhancements to Speed, Accuracy, and Functionality

An efficient Burrows-Wheeler transform-based aligner for short read mapping

Fast and accurate short read alignment with hybrid hash-tree data structure

Fast and Accurate Read Alignment for Resequencing.

GenMPI: Cluster Scalable Variant Calling for Short/Long Reads Sequencing Data

FPGA Acceleration of Short Read Alignment

A High-Performance Genomic Accelerator for Accurate Sequence-to-Graph Alignment Using Dynamic Programming Algorithm

HQAlign: Aligning nanopore reads for SV detection using current-level modeling

Recent advances in selective estrogen receptor modulators for breast cancer.

Shifted Hamming distance: a fast and accurate SIMD-friendly filter to accelerate alignment verification in read mapping

A FPGA-Based High Performance Acceleration Platform for the Next Generation Long Read Mapping

AGAThA: Fast and Efficient GPU Acceleration of Guided Sequence Alignment for Long Read Mapping

Perm: Efficient Mapping of Short Sequencing Reads with Periodic Full Sensitive Spaced Seeds

Acceleration of the long read mapping on a PC-FPGA architecture (abstract only).

Minimap2: pairwise alignment for nucleotide sequences

RASSA: Resistive Pre-Alignment Accelerator for Approximate DNA Long Read Mapping

Fast and accurate long-read alignment with Burrows–Wheeler transform

Pacybara: accurate long-read sequencing for barcoded mutagenized allelic libraries