Abstract:We prove, using the subspace embedding guarantee in a black box way, that one can achieve the spectral norm guarantee for approximate matrix multiplication with a dimensionality-reducing map having $m = O(\tilde{r}/\varepsilon^2)$ rows. Here $\tilde{r}$ is the maximum stable rank, i.e. squared ratio of Frobenius and operator norms, of the two matrices being multiplied. This is a quantitative improvement over previous work of [MZ11, KVZ14], and is also optimal for any oblivious dimensionality-reducing map. Furthermore, due to the black box reliance on the subspace embedding property in our proofs, our theorem can be applied to a much more general class of sketching matrices than what was known before, in addition to achieving better bounds. For example, one can apply our theorem to efficient subspace embeddings such as the Subsampled Randomized Hadamard Transform or sparse subspace embeddings, or even with subspace embedding constructions that may be developed in the future. Our main theorem, via connections with spectral error matrix multiplication shown in prior work, implies quantitative improvements for approximate least squares regression and low rank approximation. Our main result has also already been applied to improve dimensionality reduction guarantees for $k$-means clustering [CEMMP14], and implies new results for nonparametric regression [YPW15]. We also separately point out that the proof of the "BSS" deterministic row-sampling result of [BSS12] can be modified to show that for any matrices $A, B$ of stable rank at most $\tilde{r}$, one can achieve the spectral norm guarantee for approximate matrix multiplication of $A^T B$ by deterministically sampling $O(\tilde{r}/\varepsilon^2)$ rows that can be found in polynomial time. The original result of [BSS12] was for rank instead of stable rank. Our observation leads to a stronger version of a main theorem of [KMST10].

Improved matrix algorithms via the Subsampled Randomized Hadamard Transform

Least Squares Approximation Via Sparse Subsampled Randomized Hadamard Transform

Optimal approximate matrix product in terms of stable rank

Efficient Randomized Algorithms for the Fixed-Precision Low-Rank Matrix Approximation

Using a Non-Commutative Bernstein Bound to Approximate Some Matrix Algorithms in the Spectral Norm

Low-Rank Updates of Matrix Square Roots

On the Unreasonable Effectiveness of Single Vector Krylov Methods for Low-Rank Approximation

Randomized algorithms for low-rank matrix approximation: Design, analysis, and applications

Near-optimal hierarchical matrix approximation from matrix-vector products

Randomized Nyström approximation of non-negative self-adjoint operators

An Efficient Randomized Algorithm for Computing the Approximate Tucker Decomposition

Randomized low-rank approximations beyond Gaussian random matrices

A general error analysis for randomized low-rank approximation with application to data assimilation

A multilinear Nyström algorithm for low-rank approximation of tensors in Tucker format

Making the Nyström method highly accurate for low-rank approximations

Optimal Randomized First-Order Methods for Least-Squares Problems

Randomized Matrix Approximation to Enhance Regularized Projection Schemes in Inverse Problems

Theoretical and Experimental Analysis of a Randomized Algorithm for Sparse Fourier Transform Analysis

Faster Linear Systems and Matrix Norm Approximation via Multi-level Sketched Preconditioning