Abstract:Background: New high throughput pyrosequencers such as the 454 Life Sciences GS 20 are capable of massively parallelizing DNA sequencing providing an unprecedented rate of output data as well as potentially reducing costs. However, these new pyrosequencers bear a different error profile and provide shorter reads than those of a more traditional Sanger sequencer. These facts pose new challenges regarding how the data are handled and analyzed, in addition, the steep increase in the sequencers throughput calls for much computation power at a low cost. Results: To address these challenges, we created an automated multi-step computation pipeline integrated with a database storage system. This allowed us to store, handle, index and search (1) the output data from the GS20 sequencer (2) analysis projects, possibly multiple on every dataset (3) final results of analysis computations (4) intermediate results of computations (these allow hand-made comparisons and hence further searches by the biologists). Repeatability of computations was also a requirement. In order to access the needed computation power, we ported the pipeline to the European Grid: a large community of clusters, load balanced as a whole. In order to better achieve this Grid port we created Vnas: an innovative Grid job submission, virtual sandbox manager and job callback framework. After some runs of the pipeline aimed at tuning the parameters and thresholds for optimal results, we successfully analyzed 273 sequenced amplicons from a cancerous human sample and correctly found punctual mutations confirmed by either Sanger resequencing or NCBI dbSNP. The sequencing was performed with our 454 Life Sciences GS 20 pyrosequencer. Conclusion: We handled the steep increase in throughput from the new pyrosequencer by building an automated computation pipeline associated with database storage, and by leveraging the computing power of the European Grid. The Grid platform offers a very cost effective choice for uneven workloads, typical in many scientific research fields, provided its peculiarities can be accepted (these are discussed). The mentioned infrastructure was used to analyze human amplicons for mutations. More analyses will be performed in the future.

Enabling data and compute intensive workflows in bioinformatics

A cost-effective approach to improving performance of big genomic data analyses in clouds

Bioportal: A Portal For Deployment Of Bioinformatics Applications On Cluster And Grid Environments

Workflow-as-a-Service Cloud Platform and Deployment of Bioinformatics Workflow Applications

Enabling the democratization of the genomics revolution with a fully integrated web-based bioinformatics platform

GEO-WMS: an Improved Approach to Geoscientific Workflow Management System on HPC

Closha 2.0: a bio-workflow design system for massive genome data analysis on high performance cluster infrastructure

Efficient and Reliable Data Management for Biomedical Applications

Workflow environments for advanced cyberinfrastructure platforms

The Pegasus workflow management system: Translational computer science in practice

Astronomical Image Processing at Scale With Pegasus and Montage

Engineering Data Processing Workflows

Data handling strategies for high throughput pyrosequencers

Computational Strategies for Scalable Genomics Analysis

Computing Platforms for Big Biological Data Analytics: Perspectives and Challenges

Enabling Large Scale Scientific Computations for Expressed Sequence Tag Sequencing over Grid and Cloud Computing Clusters

A graphical, interactive and GPU-enabled workflow to process long-read sequencing data

Enhanced Usability of Managing Workflows in an Industrial Data Gateway

An Event-Driven Architecture for Genomics-Based Diagnostic Data Processing

The future of scientific workflows

Combining Hadoop with MPI to Solve Metagenomics Problems That Are Both Data- and Compute-intensive