Abstract:We prove, using the subspace embedding guarantee in a black box way, that one can achieve the spectral norm guarantee for approximate matrix multiplication with a dimensionality-reducing map having $m = O(\tilde{r}/\varepsilon^2)$ rows. Here $\tilde{r}$ is the maximum stable rank, i.e. squared ratio of Frobenius and operator norms, of the two matrices being multiplied. This is a quantitative improvement over previous work of [MZ11, KVZ14], and is also optimal for any oblivious dimensionality-reducing map. Furthermore, due to the black box reliance on the subspace embedding property in our proofs, our theorem can be applied to a much more general class of sketching matrices than what was known before, in addition to achieving better bounds. For example, one can apply our theorem to efficient subspace embeddings such as the Subsampled Randomized Hadamard Transform or sparse subspace embeddings, or even with subspace embedding constructions that may be developed in the future. Our main theorem, via connections with spectral error matrix multiplication shown in prior work, implies quantitative improvements for approximate least squares regression and low rank approximation. Our main result has also already been applied to improve dimensionality reduction guarantees for $k$-means clustering [CEMMP14], and implies new results for nonparametric regression [YPW15]. We also separately point out that the proof of the "BSS" deterministic row-sampling result of [BSS12] can be modified to show that for any matrices $A, B$ of stable rank at most $\tilde{r}$, one can achieve the spectral norm guarantee for approximate matrix multiplication of $A^T B$ by deterministically sampling $O(\tilde{r}/\varepsilon^2)$ rows that can be found in polynomial time. The original result of [BSS12] was for rank instead of stable rank. Our observation leads to a stronger version of a main theorem of [KMST10].

Probabilistic bounds on best rank-1 approximation ratio

Maximum relative distance between real rank-two and rank-one tensors

The Bounds of Restricted Isometry Constants for Low Rank Matrices Recovery

Approximate Real Symmetric Tensor Rank

Discreteness of asymptotic tensor ranks

On the rank and the approximation of symmetric tensors

Quasi-linear relation between partition and analytic rank

Z-Eigenvalue Localization Sets for Tensors and the Applications in Rank-One Approximation and Quantum Entanglement

Subrank and optimal reduction of scalar multiplications to generic tensors

Subtracting a best rank-1 approximation may increase tensor rank

What is the probability that a random symmetric tensor is close to rank-one?

A Gap in the Subrank of Tensors

On the ranks and border ranks of symmetric tensors

Random restrictions of high-rank tensors and polynomial maps

Asymptotic tensor rank is characterized by polynomials

A note on the gap between rank and border rank

Outliers and bounded rank perturbation for non-Hermitian random band matrices

Symmetric Rank-1 Approximation of Symmetric High-Order Tensors.

The spectrum of a tensor of random and deterministic matrices

Entrywise tensor-train approximation of large tensors via random embeddings

Optimal approximate matrix product in terms of stable rank