Abstract:This paper describes efficient algorithms for computing rank-revealing factorizations of matrices that are too large to fit in RAM, and must instead be stored on slow external memory devices such as solid-state or spinning disk hard drives (out-of-core or out-of-memory). Traditional algorithms for computing rank revealing factorizations, such as the column pivoted QR factorization, or techniques for computing a full singular value decomposition of a matrix, are very communication intensive. They are naturally expressed as a sequence of matrix-vector operations, which become prohibitively expensive when data is not available in main memory. Randomization allows these methods to be reformulated so that large contiguous blocks of the matrix can be processed in bulk. The paper describes two distinct methods. The first is a blocked version of column pivoted Householder QR, organized as a "left-looking" method to minimize the number of write operations (which are more expensive than read operations on a spinning disk drive). The second method results in a so called UTV factorization which expresses a matrix $A$ as $A = U T V^*$ where $U$ and $V$ are unitary, and $T$ is triangular. This method is organized as an algorithm-by-blocks, in which floating point operations overlap read and write operations. The second method incorporates power iterations, and is exceptionally good at revealing the numerical rank; it can often be used as a substitute for a full singular value decomposition. Numerical experiments demonstrate that the new algorithms are almost as fast when processing data stored on a hard drive as traditional algorithms are for data stored in main memory. To be precise, the computational time for fully factorizing an $n\times n$ matrix scales as $cn^{3}$, with a scaling constant $c$ that is only marginally larger when the matrix is stored out of core.

Butterfly factorization via randomized matrix-vector multiplications

Butterfly factorization with error guarantees

Efficient Identification of Butterfly Sparse Matrix Factorizations

Butterfly factorization by algorithmic identification of rank-one blocks

Interpolative Butterfly Factorization

Sparse Approximate Multifrontal Factorization with Butterfly Compression for High Frequency Wave Equations

Stochastic Subsampling for Factorizing Huge Matrices

Growth factors of random butterfly matrices and the stability of avoiding pivoting

Generalizing Random Butterfly Transforms to Arbitrary Matrix Sizes

Binary Matrix Factorisation via Column Generation

The Time Complexity of Fully Sparse Matrix Multiplication

A Note on Randomized Element-wise Matrix Sparsification

Randomized Algorithms for Symmetric Nonnegative Matrix Factorization

Iterative methods for solving factorized linear systems

Randomization or Condensation?

Sparse Factorization of Large Square Matrices

Randomized Strong Recursive Skeletonization: Simultaneous compression and factorization of $\mathcal{H}$-matrices in the Black-Box Setting

Computing rank-revealing factorizations of matrices stored out-of-core

Fast $(1+\varepsilon)$-Approximation Algorithms for Binary Matrix Factorization

Parallel Approximate Matrix Factorization For Kernel Methods

Randomization or Condensation?: Linear-Cost Matrix Sketching Via Cascaded Compression Sampling.