Abstract:Computational analysis of a large number of genomes requires a data structure that can represent the dataset compactly while also enabling efficient operations on variants and samples. Current practice is to store large-scale genetic polymorphism data using tabular data structures and file formats, where rows and columns represent samples and genetic variants. However, encoding genetic data in such formats has become unsustainable. For example, the UK Biobank polymorphism data of 200,000 phased whole genomes has exceeded 350 terabytes (TB) in Variant Call Format (VCF), cumbersome and inefficient to work with. To mitigate the computational burden, we introduce the Genotype Representation Graph (GRG), an extremely compact data structure to losslessly present phased whole-genome polymorphisms. A GRG is a fully connected hierarchical graph that exploits variant-sharing across samples, leveraging ideas inspired by Ancestral Recombination Graphs. Capturing variant-sharing in a multitree structure compresses biobank-scale human data to the point where it can fit in a typical server'ss RAM (5-26 gigabytes (GB) per chromosome), and enables graph-traversal algorithms to trivially reuse computed values, both of which can significantly reduce computation time. We have developed a command-line tool and a library usable via both C++ and Python for constructing and processing GRG files which scales to a million whole genomes. It takes 160GB disk space to encode the information in 200,000 UK Biobank phased whole genomes as a GRG, more than 13 times smaller than the size of compressed VCF. We show that summaries of genetic variants such as allele frequency and association effect can be computed on GRG via graph traversal that runs significantly faster than all tested alternatives, including vcf.gz, PLINK BED, tree sequence, XSI, and Savvy. Furthermore, GRG is particularly suitable for doing repeated calculations and interactive data analysis. We anticipate that GRG-based algorithms will improve the scalability of various types of computation and generally lower the cost of analyzing large genomic datasets.

GW: ultra-fast chromosome-scale visualisation of genomics data

The 3D Genome Browser: a Web-Based Browser for Visualizing 3D Genome Organization and Long-Range Chromatin Interactions

The Genome Explorer Genome Browser

Cisgenome Browser: A Flexible Tool for Genomic Data Visualization

Integrative Genomics Viewer (IGV): high-performance genomics data visualization and exploration

A Web-Based Multi-Genome Synteny Viewer for Customized Data

GBC: a parallel toolkit based on highly addressable byte-encoding blocks for extremely large-scale genotypes of species

Using WebGBrowse to Visualize Genome Annotation on GBrowse.

Glbase: a Framework for Combining, Analyzing and Displaying Heterogeneous Genomic and High-Throughput Sequencing Data

The Personal Genome Browser: Visualizing Functions of Genetic Variants

GobyWeb: simplified management and analysis of gene expression and DNA methylation sequencing data

WikiGWA: an Open Platform for Collecting and Using Genome-Wide Association Results

The Genomic HyperBrowser: inferential genomics at the sequence level

HiGlass: web-based visual exploration and analysis of genome interaction maps

3D-GNOME 3.0: a three-dimensional genome modelling engine for analysing changes of promoter-enhancer contacts in the human genome

Genotype Representation Graphs: Enabling Efficient Analysis of Biobank-Scale Data

HiView: an Integrative Genome Browser to Leverage Hi-C Results for the Interpretation of GWAS Variants

GWGGI: software for genome-wide gene-gene interaction analysis

gghic: A Versatile R Package for Exploring and Visualizing 3D Genome Organization

High-performance Genomic Analysis Framework with In-Memory Computing

Efficient indexing and querying of annotations in a pangenome graph