khmer: Working with Big Data in Bioinformatics

Eric McDonald,C. Titus Brown

DOI: https://doi.org/10.48550/arXiv.1303.2223

2013-03-09

Abstract:We introduce design and optimization considerations for the 'khmer' package.

Computational Engineering, Finance, and Science,Genomics

What problem does this paper attempt to address?

Walking the Talk: Adopting and Adapting Sustainable Scientific Software Development processes in a Small Biology Lab

Michael R. Crusoe,C. Titus Brown

DOI: https://doi.org/10.5334/jors.35

2016-11-29

Journal of Open Research Software

Abstract:The khmer software project provides both research and production functionality for largescale nucleic-acid sequence analysis. The software implements several novel data structures and algorithms that perform data pre-fltering for common bioinformatics tasks, including sequence mapping and de novo assembly. Development is driven by a small lab with one full-time developer (MRC), as well as several graduate students and a professor (CTB) who contribute regularly to research features. Here we describe our efforts to bring better design, testing, and more open development to the khmer software project as of version 1.1. The khmer software is developed openly at http://github.com/dib-lab/khmer/.
The HTPmod Shiny application enables modeling and visualization of large-scale biological data

Dijun Chen,Liang-Yu Fu,Dahui Hu,Christian Klukas,Ming Chen,Kerstin Kaufmann

DOI: https://doi.org/10.1038/s42003-018-0091-x

IF: 6.548

2018-01-01

Communications Biology

Abstract:The wave of high-throughput technologies in genomics and phenomics are enabling data to be generated on an unprecedented scale and at a reasonable cost. Exploring the large-scale data sets generated by these technologies to derive biological insights requires efficient bioinformatic tools. Here we introduce an interactive, open-source web application (HTPmod) for high-throughput biological data modeling and visualization. HTPmod is implemented with the Shiny framework by integrating the computational power and professional visualization of R and including various machine-learning approaches. We demonstrate that HTPmod can be used for modeling and visualizing large-scale, high-dimensional data sets (such as multiple omics data) under a broad context. By reinvestigating example data sets from recent studies, we find not only that HTPmod can reproduce results from the original studies in a straightforward fashion and within a reasonable time, but also that novel insights may be gained from fast reinvestigation of existing data by HTPmod.
Biological Big Bytes: Integrative Analysis of Large Biological Datasets

Ming Chen,Andrew Harrison,Hugh Shanahan,Yuriy Orlov

DOI: https://doi.org/10.1515/jib-2017-0052

2017-01-01

Journal of Integrative Bioinformatics

Abstract:1 Department of Bioinformatics, College of Life Sciences, Zhejiang University, Hangzhou, P.R. China, E-mail: mchen@zju.edu.cn 2 Department of Mathematical Sciences, University of Essex, Essex, England, UK 3 Department of Computer Science, Royal Holloway, University of London, Egham, England, UK 4 Institute of Cytology and Genetics SB RAS, Novosibirsk, Russia 5 Novosibirsk State University, Novosibirsk, Russia
These are not the k-mers you are looking for: efficient online k-mer counting using a probabilistic data structure

Qingpeng Zhang,Jason Pell,Rosangela Canino-Koning,Adina Chuang Howe,C. Titus Brown

DOI: https://doi.org/10.1371/journal.pone.0101271

2014-07-15

Abstract:K-mer abundance analysis is widely used for many purposes in nucleotide sequence analysis, including data preprocessing for de novo assembly, repeat detection, and sequencing coverage estimation. We present the khmer software package for fast and memory efficient online counting of k-mers in sequencing data sets. Unlike previous methods based on data structures such as hash tables, suffix arrays, and trie structures, khmer relies entirely on a simple probabilistic data structure, a Count-Min Sketch. The Count-Min Sketch permits online updating and retrieval of k-mer counts in memory which is necessary to support online k-mer analysis algorithms. On sparse data sets this data structure is considerably more memory efficient than any exact data structure. In exchange, the use of a Count-Min Sketch introduces a systematic overcount for k-mers; moreover, only the counts, and not the k-mers, are stored. Here we analyze the speed, the memory usage, and the miscount rate of khmer for generating k-mer frequency distributions and retrieving k-mer counts for individual k-mers. We also compare the performance of khmer to several other k-mer counting packages, including Tallymer, Jellyfish, BFCounter, DSK, KMC, Turtle and KAnalyze. Finally, we examine the effectiveness of profiling sequencing error, k-mer abundance trimming, and digital normalization of reads in the context of high khmer false positive rates. khmer is implemented in C++ wrapped in a Python interface, offers a tested and robust API, and is freely available under the BSD license at <a class="link-external link-http" href="http://github.com/ged-lab/khmer" rel="external noopener nofollow">this http URL</a>.

Genomics
Analyzing big datasets of genomic sequences: fast and scalable collection of k-mer statistics

Umberto Ferraro Petrillo,Mara Sorella,Giuseppe Cattaneo,Raffaele Giancarlo,Simona E. Rombo

DOI: https://doi.org/10.1186/s12859-019-2694-8

IF: 3.307

2019-04-01

BMC Bioinformatics

Abstract:<h3>Background</h3>Distributed approaches based on the MapReduce programming paradigm have started to be proposed in the Bioinformatics domain, due to the large amount of data produced by the next-generation sequencing techniques. However, the use of MapReduce and related Big Data technologies and frameworks (e.g., Apache Hadoop and Spark) does not necessarily produce satisfactory results, in terms of both efficiency and effectiveness. We discuss how the development of distributed and Big Data management technologies has affected the analysis of large datasets of biological sequences. Moreover, we show how the choice of different parameter configurations and the careful engineering of the software with respect to the specific framework under consideration may be crucial in order to achieve good performance, especially on very large amounts of data. We choose k-mers counting as a case study for our analysis, and Spark as the framework to implement FastKmer, a novel approach for the extraction of k-mer statistics from large collection of biological sequences, with arbitrary values of k.

biochemical research methods,biotechnology & applied microbiology,mathematical & computational biology
A Suite of Tools for Biologists That Improve Accessibility and Visualization of Large Systems Genetics Datasets: Applications to the Hybrid Mouse Diversity Panel

Christoph D Rau,Mete Civelek,Calvin Pan,Aldons J Lusis

DOI: https://doi.org/10.1007/978-1-4939-6427-7_7

Abstract:In this chapter we address the recent explosion in large multilevel population studies such as the METSIM study in humans as well as large panels of animal models such as the Hybrid Mouse Diversity Panel or the BXD set of recombinant inbred strains. These studies have harnessed the increasing affordability of large-scale high-throughput profiling to gather massive quantities of data. These datasets, spread across different -omics levels (genome, transcriptome, etc.), different tissues (e.g. heart, plasma, bone) and different environmental factors (e.g. diet, drugs) each individually have led to a number of novel findings relevant to a variety of complex diseases and other phenotypes. The analysis of these results, however, is often limited to individuals with a comprehensive understanding of database languages such as SQL. In this chapter, we describe the development of a GUI-based database analysis suite, using the Hybrid Mouse Diversity Panel as an example to lay out a series of methods for visualization and integration of large systems genetics datasets. The database is based on the Shiny suite of tools in R, and is transferrable to other SQL-based datasets.
Big data analytics in bioinformatics: architectures, techniques, tools and issues

Hirak Kashyap,Hasin Afzal Ahmed,Nazrul Hoque,Swarup Roy,Dhruba Kumar Bhattacharyya

DOI: https://doi.org/10.1007/s13721-016-0135-4

2016-09-01

Network Modeling Analysis in Health Informatics and Bioinformatics

Abstract:Bioinformatics research is characterized by voluminous and incremental datasets and complex data analytics methods. The machine learning methods used in bioinformatics are iterative and parallel. These methods can be scaled to handle big data using the distributed and parallel computing technologies. Usually big data tools perform computation in batch mode and are not optimized for iterative processing and high data dependency among operations. In the recent years, parallel, incremental, and multi-view machine learning algorithms have been proposed. Similarly, graph-based architectures and in-memory big data tools have been developed to minimize I/O cost and optimize iterative processing. However, standard big data architectures are still lacking. Also appropriate tools are not available for many important bioinformatics problems, such as fast construction of co-expression and regulatory networks and salient module identification, detection of complexes over growing protein-protein interaction data, fast analysis of massive DNA, RNA, and protein sequence data, and fast querying on incremental and heterogeneous disease networks. This paper addresses the issues and challenges posed by several big data problems in bioinformatics, and gives an overview of the state of the art and the future research opportunities.
Big Data Analytics in Bioinformatics: A Machine Learning Perspective

Hirak Kashyap,Hasin Afzal Ahmed,Nazrul Hoque,Swarup Roy,Dhruba Kumar Bhattacharyya

DOI: https://doi.org/10.48550/arXiv.1506.05101

2015-06-15

Computational Engineering, Finance, and Science

Abstract:Bioinformatics research is characterized by voluminous and incremental datasets and complex data analytics methods. The machine learning methods used in bioinformatics are iterative and parallel. These methods can be scaled to handle big data using the distributed and parallel computing technologies. Usually big data tools perform computation in batch-mode and are not optimized for iterative processing and high data dependency among operations. In the recent years, parallel, incremental, and multi-view machine learning algorithms have been proposed. Similarly, graph-based architectures and in-memory big data tools have been developed to minimize I/O cost and optimize iterative processing. However, there lack standard big data architectures and tools for many important bioinformatics problems, such as fast construction of co-expression and regulatory networks and salient module identification, detection of complexes over growing protein-protein interaction data, fast analysis of massive DNA, RNA, and protein sequence data, and fast querying on incremental and heterogeneous disease networks. This paper addresses the issues and challenges posed by several big data problems in bioinformatics, and gives an overview of the state of the art and the future research opportunities.
Empowering biologists to decode omics data: the Genekitr R package and web server

Yunze Liu,Gang Li

DOI: https://doi.org/10.1186/s12859-023-05342-9

IF: 3.307

2023-05-23

BMC Bioinformatics

Abstract:Background: A variety of high-throughput analyses, such as transcriptome, proteome, and metabolome analysis, have been developed, producing unprecedented amounts of omics data. These studies generate large gene lists, of which the biological significance shall be deeply understood. However, manually interpreting these lists is difficult, especially for non-bioinformatics-savvy scientists. Results: We developed an R package and a corresponding web server-Genekitr, to assist biologists in exploring large gene sets. Genekitr comprises four modules: gene information retrieval, ID (identifier) conversion, enrichment analysis and publication-ready plotting. Currently, the information retrieval module can retrieve information on up to 23 attributes for genes of 317 organisms. The ID conversion module assists in ID-mapping of genes, probes, proteins, and aliases. The enrichment analysis module organizes 315 gene set libraries in different biological contexts by over-representation analysis and gene set enrichment analysis. The plotting module performs customizable and high-quality illustrations that can be used directly in presentations or publications. Conclusions: This web server tool will make bioinformatics more accessible to scientists who might not have programming expertise, allowing them to perform bioinformatics tasks without coding.
GECKO is a genetic algorithm to classify and explore high throughput sequencing data

Aubin Thomas,Sylvain Barriere,Lucile Broseus,Julie Brooke,Claudio Lorenzi,Jean-Philippe Villemin,Gregory Beurier,Robert Sabatier,Christelle Reynes,Alban Mancheron,William Ritchie

DOI: https://doi.org/10.1038/s42003-019-0456-9

IF: 6.548

2019-06-20

Communications Biology

Abstract:Comparative analysis of high throughput sequencing data between multiple conditions often involves mapping of sequencing reads to a reference and downstream bioinformatics analyses. Both of these steps may introduce heavy bias and potential data loss. This is especially true in studies where patient transcriptomes or genomes may vary from their references, such as in cancer. Here we describe a novel approach and associated software that makes use of advances in genetic algorithms and feature selection to comprehensively explore massive volumes of sequencing data to classify and discover new sequences of interest without a mapping step and without intensive use of specialized bioinformatics pipelines. We demonstrate that our approach called GECKO for GEnetic Classification using k-mer Optimization is effective at classifying and extracting meaningful sequences from multiple types of sequencing approaches including mRNA, microRNA, and DNA methylome data.

biology
Big Data access and infrastructure for modern biology: case studies in data repository utility

Nathan C Boles,Tyler Stone,Charles Bergeron,Thomas R Kiehl

DOI: https://doi.org/10.1111/nyas.13281

Abstract:Big Data is no longer solely the purview of big organizations with big resources. Today's routine tools and experimental methods can generate large slices of data. For example, high-throughput sequencing can quickly interrogate biological systems for the expression levels of thousands of different RNAs, examine epigenetic marks throughout the genome, and detect differences in the genomes of individuals. Multichannel electrophysiology platforms produce gigabytes of data in just a few minutes of recording. Imaging systems generate videos capturing biological behaviors over the course of days. Thus, any researcher now has access to a veritable wealth of data. However, the ability of any given researcher to utilize that data is limited by her/his own resources and skills for downloading, storing, and analyzing the data. In this paper, we examine the necessary resources required to engage Big Data, survey the state of modern data analysis pipelines, present a few data repository case studies, and touch on current institutions and programs supporting the work that relies on Big Data.
Design and Construction of a Big Data Analytics Framework for Health Applications

Mu-Hsing Kuo,Dillon Chrimes,Belaid Moa,Wei Hu

DOI: https://doi.org/10.1109/smartcity.2015.140

2015-01-01

Abstract:We propose to establish a framework for supporting Big Data Analytics (BDA) on real healthcare big data. To test the analytic framework, we used UVic WestGrid (4412 cores computer cluster) to analyze the emulation of 10 billion healthcare records that represented the main hospital system and its reporting via its data warehouse stored at Vancouver Island Health Authority (VIHA). The study showed that the build of the BDA platform requires changes to the configurations to the MapReduce component of Hadoop (HDFS) and to the indexing of HBASE. The ingestion and replication of the data over a large volume iteratively offers a method for data migration of large volumes of real healthcare data via HDFS and to query in that some distributed filing system. Furthermore, the query performance was very satisfied via Apache Phoenix layer that is run in parallel across all nodes on HBASE. The study has demonstrated that the proposed BDA process and configuration met patient data security and performance requirements of healthcare BDA.
Workflows for microarray data processing in the Kepler environment

Thomas Stropp,Timothy McPhillips,Bertram Ludäscher,Mark Bieda

DOI: https://doi.org/10.1186/1471-2105-13-102

IF: 3.307

2012-05-17

BMC Bioinformatics

Abstract:Background: Microarray data analysis has been the subject of extensive and ongoing pipeline development due to its complexity, the availability of several options at each analysis step, and the development of new analysis demands, including integration with new data sources. Bioinformatics pipelines are usually custom built for different applications, making them typically difficult to modify, extend and repurpose. Scientific workflow systems are intended to address these issues by providing general-purpose frameworks in which to develop and execute such pipelines. The Kepler workflow environment is a well-established system under continual development that is employed in several areas of scientific research. Kepler provides a flexible graphical interface, featuring clear display of parameter values, for design and modification of workflows. It has capabilities for developing novel computational components in the R, Python, and Java programming languages, all of which are widely used for bioinformatics algorithm development, along with capabilities for invoking external applications and using web services. Results: We developed a series of fully functional bioinformatics pipelines addressing common tasks in microarray processing in the Kepler workflow environment. These pipelines consist of a set of tools for GFF file processing of NimbleGen chromatin immunoprecipitation on microarray (ChIP-chip) datasets and more comprehensive workflows for Affymetrix gene expression microarray bioinformatics and basic primer design for PCR experiments, which are often used to validate microarray results. Although functional in themselves, these workflows can be easily customized, extended, or repurposed to match the needs of specific projects and are designed to be a toolkit and starting point for specific applications. These workflows illustrate a workflow programming paradigm focusing on local resources (programs and data) and therefore are close to traditional shell scripting or R/BioConductor scripting approaches to pipeline design. Finally, we suggest that microarray data processing task workflows may provide a basis for future example-based comparison of different workflow systems. Conclusions: We provide a set of tools and complete workflows for microarray data analysis in the Kepler environment, which has the advantages of offering graphical, clear display of conceptual steps and parameters and the ability to easily integrate other resources such as remote data and web services.
Closha 2.0: a bio-workflow design system for massive genome data analysis on high performance cluster infrastructure

Gunhwan Ko,Pan-Gyu Kim,Byung-Ha Yoon,JaeHee Kim,Wangho Song,IkSu Byeon,JongCheol Yoon,Byungwook Lee,Young-Kuk Kim

DOI: https://doi.org/10.1186/s12859-024-05963-8

IF: 3.307

2024-11-19

BMC Bioinformatics

Abstract:The explosive growth of next-generation sequencing data has resulted in ultra-large-scale datasets and significant computational challenges. As the cost of next-generation sequencing (NGS) has decreased, the amount of genomic data has surged globally. However, the cost and complexity of the computational resources required continue to be substantial barriers to leveraging big data. A promising solution to these computational challenges is cloud computing, which provides researchers with the necessary CPUs, memory, storage, and software tools.

biochemical research methods,biotechnology & applied microbiology,mathematical & computational biology
TBtools: An Integrative Toolkit Developed for Interactive Analyses of Big Biological Data

Chengjie Chen,Hao Chen,Yi Zhang,Hannah R Thomas,Margaret H Frank,Yehua He,Rui Xia,Hannah R. Thomas,Margaret H. Frank

DOI: https://doi.org/10.1016/j.molp.2020.06.009

IF: 27.5

2020-08-01

Molecular Plant

Abstract:The rapid development of high-throughput sequencing techniques has led biology into the big-data era. Data analyses using various bioinformatics tools rely on programming and command-line environments, which are challenging and time-consuming for most wet-lab biologists. Here, we present TBtools (a Toolkit for Biologists integrating various biological data-handling tools), a stand-alone software with a user-friendly interface. The toolkit incorporates over 130 functions, which are designed to meet the increasing demand for big-data analyses, ranging from bulk sequence processing to interactive data visualization. A wide variety of graphs can be prepared in TBtools using a new plotting engine ("JIGplot") developed to maximize their interactive ability; this engine allows quick point-and-click modification of almost every graphic feature. TBtools is platform-independent software that can be run under all operating systems with Java Runtime Environment 1.6 or newer. It is freely available to non-commercial users at <a href="https://github.com/CJ-Chen/TBtools/releases">https://github.com/CJ-Chen/TBtools/releases</a>.

biochemistry & molecular biology,plant sciences
KDE Bioscience: Platform for Bioinformatics Analysis Workflows.

Qiang Lu,Pei Hao,Vasa Curcin,Weizhong He,Yuan-Yuan Li,Qing-Ming Luo,Yi-Ke Guo,Yi-Xue Li

DOI: https://doi.org/10.1016/j.jbi.2005.09.001

IF: 8

2006-01-01

Journal of Biomedical Informatics

Abstract:Bioinformatics is a dynamic research area in which a large number of algorithms and programs have been developed rapidly and independently without much consideration so far of the need for standardization. The lack of such common standards combined with unfriendly interfaces make it difficult for biologists to learn how to use these tools and to translate the data formats from one to another. Consequently, the construction of an integrative bioinformatics platform to facilitate biologists' research is an urgent and challenging task. KDE Bioscience is a java-based software platform that collects a variety of bioinformatics tools and provides a workflow mechanism to integrate them. Nucleotide and protein sequences from local flat files, web sites, and relational databases can be entered, annotated, and aligned. Several home-made or 3rd-party viewers are built-in to provide visualization of annotations or alignments. KDE Bioscience can also be deployed in client-server mode where simultaneous execution of the same workflow is supported for multiple users. Moreover, workflows can be published as web pages that can be executed from a web browser. The power of KDE Bioscience comes from the integrated algorithms and data sources. With its generic workflow mechanism other novel calculations and simulations can be integrated to augment the current sequence analysis functions. Because of this flexible and extensible architecture, KDE Bioscience makes an ideal integrated informatics environment for future bioinformatics or systems biology research.
BDViewer — A Web-Based Big Data Processing and Visualization Tool

Yan Li,Junming Ma,Bo An,Donggang Cao

DOI: https://doi.org/10.1109/COMPSAC.2018.00080

2018-01-01

Abstract:The size of data sets being collected and analyzed in data science field is growing rapidly, making traditional big data processing solution prohibitively expensive. Especially when the data sets are too large, distributed techniques are inevitable even for simple embarrassing parallel jobs. However, distributed computing is still inaccessible to a large number of users. For example, many average users are still struggling with complex cluster management and configuration tools[24] even just for summing up a group of numbers in a large data file. In this paper, we present BDViewer, a web-based big data processing and visualizing tool. BDViewer uses JavaScript plugins to enable users to view, process and visualize their large data files just through a web browser. By just clicking a button, users can open a large data file online and view the file contents immediately no matter how large the file is. In the back-end, BDViewer is built on a virtual private cloud system. Users' operations in a web browser are converted into map-reduce jobs and MPI tasks that are executed on the cloud. At the end of this paper, some experiments are carried out, which demonstrate BDViewer's effectiveness and ease of use.
BRISK--research-oriented storage kit for biology-related data

Alan Tan,Ben Tripp,Denise Daley

DOI: https://doi.org/10.1093/bioinformatics/btr389

IF: 5.8

2011-09-01

Bioinformatics

Abstract:Motivation: In genetic science, large-scale international research collaborations represent a growing trend. These collaborations have demanding and challenging database, storage, retrieval and communication needs. These studies typically involve demographic and clinical data, in addition to the results from numerous genomic studies (omics studies) such as gene expression, eQTL, genome-wide association and methylation studies, which present numerous challenges, thus the need for data integration platforms that can handle these complex data structures. Inefficient methods of data transfer and access control still plague research collaboration. As science becomes more and more collaborative in nature, the need for a system that adequately manages data sharing becomes paramount. Results: Biology-Related Information Storage Kit (BRISK) is a package of several web-based data management tools that provide a cohesive data integration and management platform. It was specifically designed to provide the architecture necessary to promote collaboration and expedite data sharing between scientists. Availability and implementation: The software, documentation, Java source code and demo are available at http://genapha.icapture.ubc.ca/brisk/index.jsp. BRISK was developed in Java, and tested on an Apache Tomcat 6 server with a MySQL database. Contact: denise.daley@hli.ubc.ca.
Guide to k-mer approaches for genomics across the tree of life

Katharine M. Jenike,Lucía Campos-Domínguez,Marilou Boddé,José Cerca,Christina N. Hodson,Michael C. Schatz,Kamil S. Jaron

2024-04-02

Abstract:The wide array of currently available genomes display a wonderful diversity in size, composition and structure with many more to come thanks to several global biodiversity genomics initiatives starting in recent years. However, sequencing of genomes, even with all the recent advances, can still be challenging for both technical (e.g. small physical size, contaminated samples, or access to appropriate sequencing platforms) and biological reasons (e.g. germline restricted DNA, variable ploidy levels, sex chromosomes, or very large genomes). In recent years, k-mer-based techniques have become popular to overcome some of these challenges. They are based on the simple process of dividing the analysed sequences (e.g. raw reads or genomes) into a set of sub-sequences of length k, called k-mers. Despite this apparent simplicity, k-mer-based analysis allows for a rapid and intuitive assessment of complex sequencing datasets. Here, we provide the first comprehensive review to the theoretical properties and practical applications of k-mers in biodiversity genomics, serving as a reference manual for this powerful approach.

Genomics
mbctools: A User-Friendly Metabarcoding and Cross-Platform Pipeline for Analyzing Multiple Amplicon Sequencing Data across a Large Diversity of Organisms

Christian Barnabé,Guilhem Sempéré,Vincent Manzanilla,Joel Moo Millan,Antoine Amblard-Rambert,Etienne Waleckx

DOI: https://doi.org/10.1101/2024.02.08.579441

2024-09-21

Abstract:We developed a python package called , designed to offer a cross-platform tool for processing amplicon data from various organisms in the context of metabarcoding studies. It can handle the most common tasks in metabarcoding pipelines such as paired-end merging, primer trimming, quality filtering, sequence denoising, zero-radius operational taxonomic unit (ZOTU) filtering, and has the capability to process multiple genetic markers simultaneously. is a menu-driven program that eliminates the need for expertise in command-line skills and ensures documentation of each analysis for reproducibility purposes. The software, designed to run in a console, offers an interactive experience, guided by keyboard inputs, assisting users along the way through data processing and hiding the complexity of command lines by letting them concentrate on selecting parameters to apply in each step of the process. In our workflow, VSEARCH is utilized for processing files derived from amplicon-based Next-Generation Sequencing data. This software is a versatile open-source tool for processing amplicon sequences, offering advantages such as high speed, efficient memory usage, and the ability to handle large datasets. It provides functions for various tasks such as dereplication, clustering, chimera detection, and taxonomic assignment. VSEARCH is thus very efficient in retrieving the overall diversity of a sample. To adapt to the diversity of projects in metabarcoding, we facilitate the reprocessing of datasets with the possibility to adjust parameters. can also be launched in a headless mode, making it suited for integration into pipelines running on High-Performance Computing environments. is available at , .

Bioinformatics

khmer: Working with Big Data in Bioinformatics

Walking the Talk: Adopting and Adapting Sustainable Scientific Software Development processes in a Small Biology Lab

The HTPmod Shiny application enables modeling and visualization of large-scale biological data

Biological Big Bytes: Integrative Analysis of Large Biological Datasets

These are not the k-mers you are looking for: efficient online k-mer counting using a probabilistic data structure

Analyzing big datasets of genomic sequences: fast and scalable collection of k-mer statistics

A Suite of Tools for Biologists That Improve Accessibility and Visualization of Large Systems Genetics Datasets: Applications to the Hybrid Mouse Diversity Panel

Big data analytics in bioinformatics: architectures, techniques, tools and issues

Big Data Analytics in Bioinformatics: A Machine Learning Perspective

Empowering biologists to decode omics data: the Genekitr R package and web server

GECKO is a genetic algorithm to classify and explore high throughput sequencing data

Big Data access and infrastructure for modern biology: case studies in data repository utility

Design and Construction of a Big Data Analytics Framework for Health Applications

Workflows for microarray data processing in the Kepler environment

Closha 2.0: a bio-workflow design system for massive genome data analysis on high performance cluster infrastructure

TBtools: An Integrative Toolkit Developed for Interactive Analyses of Big Biological Data

KDE Bioscience: Platform for Bioinformatics Analysis Workflows.

BDViewer — A Web-Based Big Data Processing and Visualization Tool

BRISK--research-oriented storage kit for biology-related data

Guide to k-mer approaches for genomics across the tree of life

mbctools: A User-Friendly Metabarcoding and Cross-Platform Pipeline for Analyzing Multiple Amplicon Sequencing Data across a Large Diversity of Organisms