Abstract:The computation of a similarity measure for genomic data is a standard tool in computational genetics. The principal components of such matrices are routinely used to correct for biases due to confounding by population stratification, for instance in linear regressions. However, the calculation of both a similarity matrix and its singular value decomposition (SVD) are computationally intensive. The contribution of this article is threefold. First, we demonstrate that the calculation of three matrices (called the covariance matrix, the weighted Jaccard matrix, and the genomic relationship matrix) can be reformulated in a unified way which allows for the application of a randomized SVD algorithm, which is faster than the traditional computation. The fast SVD algorithm we present is adapted from an existing randomized SVD algorithm and ensures that all computations are carried out in sparse matrix algebra. The algorithm only assumes that row-wise and column-wise subtraction and multiplication of a vector with a sparse matrix is available, an operation that is efficiently implemented in common sparse matrix packages. An exception is the so-called Jaccard matrix, which does not have a structure applicable for the fast SVD algorithm. Second, an approximate Jaccard matrix is introduced to which the fast SVD computation is applicable. Third, we establish guaranteed theoretical bounds on the accuracy (in norm and angle) between the principal components of the Jaccard matrix and the ones of our proposed approximation, thus putting the proposed Jaccard approximation on a solid mathematical foundation, and derive the theoretical runtime of our algorithm. We illustrate that the approximation error is low in practice and empirically verify the theoretical runtime scalings on both simulated data and data of the 1000 Genome Project.

A Simple and Fast Algorithm for Generating Correlation Matrices with a Known Average Correlation Coefficient

Optimal low-rank approximation to a correlation matrix

A Fast Algorithm for the Computation of HAC Covariance Matrix Estimators

A Set of Efficient Methods to Generate High-Dimensional Binary Data with Specified Correlation Structures

Cleaning large correlation matrices: Tools from Random Matrix Theory

Improved Approximation and Visualization of the Correlation Matrix

Fast robust correlation for high-dimensional data

Efficient Algorithms for Positive Semi-Definite Total Least Squares Problems, Minimum Rank Problem and Correlation Matrix Computation

Riemannian statistics meets random matrix theory: towards learning from high-dimensional covariance matrices

Faddeev Approach to the Study of One- and Two-hole Spectral Functions

An Efficient Algorithm for Information Decomposition and Extraction.

Combinatorial optimization of the coefficient of determination

Measuring Linear Correlation Between Random Vectors

Kendall Correlation Coefficients for Portfolio Optimization

Compatibility and attainability of matrices of correlation-based measures of concordance

Average-Case Matrix Discrepancy: Asymptotics and Online Algorithms

Moderate Deviation Principle for the Determinant of Sample Correlation Matrix

Combinatorial Correlation Clustering

A Fast Random Sampling Algorithm for Sparsifying Matrices

Fast computation of the eigensystem of genomic similarity matrices

A Scalable Approach for General Correlation Clustering