Abstract:We prove, using the subspace embedding guarantee in a black box way, that one can achieve the spectral norm guarantee for approximate matrix multiplication with a dimensionality-reducing map having $m = O(\tilde{r}/\varepsilon^2)$ rows. Here $\tilde{r}$ is the maximum stable rank, i.e. squared ratio of Frobenius and operator norms, of the two matrices being multiplied. This is a quantitative improvement over previous work of [MZ11, KVZ14], and is also optimal for any oblivious dimensionality-reducing map. Furthermore, due to the black box reliance on the subspace embedding property in our proofs, our theorem can be applied to a much more general class of sketching matrices than what was known before, in addition to achieving better bounds. For example, one can apply our theorem to efficient subspace embeddings such as the Subsampled Randomized Hadamard Transform or sparse subspace embeddings, or even with subspace embedding constructions that may be developed in the future. Our main theorem, via connections with spectral error matrix multiplication shown in prior work, implies quantitative improvements for approximate least squares regression and low rank approximation. Our main result has also already been applied to improve dimensionality reduction guarantees for $k$-means clustering [CEMMP14], and implies new results for nonparametric regression [YPW15]. We also separately point out that the proof of the "BSS" deterministic row-sampling result of [BSS12] can be modified to show that for any matrices $A, B$ of stable rank at most $\tilde{r}$, one can achieve the spectral norm guarantee for approximate matrix multiplication of $A^T B$ by deterministically sampling $O(\tilde{r}/\varepsilon^2)$ rows that can be found in polynomial time. The original result of [BSS12] was for rank instead of stable rank. Our observation leads to a stronger version of a main theorem of [KMST10].

Randomized Algorithms for the Computation of Multilinear Rank-$$(\Mu _1,\mu _2,\mu _3)$$ Approximations

Randomized Algorithms for the Low Multilinear Rank Approximations of Tensors

The Computation of Low Multilinear Rank Approximations of Tensors Via Power Scheme and Random Projection

Randomized Algorithms for the Approximations of Tucker and the Tensor Train Decompositions.

Efficient algorithms for Tucker decomposition via approximate matrix multiplication

An Efficient Randomized Algorithm for Computing the Approximate Tucker Decomposition

Efficient Bounds and Estimates for Canonical Angles in Randomized Subspace Approximations

Randomized algorithms for computing the tensor train approximation and their applications

Efficient Alternating Least Squares Algorithms for Low Multilinear Rank Approximation of Tensors

A Random Matrix Approach to Low-Multilinear-Rank Tensor Approximation

Randomized low-rank approximations beyond Gaussian random matrices

Randomized Approximation of the Gram Matrix: Exact Computation and Probabilistic Bounds

Efficient randomized tensor-based algorithms for function approximation and low-rank kernel interactions

Fast Randomized Numerical Rank Estimation for Numerically Low-Rank Matrices

Nonconvex Robust High-Order Tensor Completion Using Randomized Low-Rank Approximation

Optimal approximate matrix product in terms of stable rank

A multilinear Nyström algorithm for low-rank approximation of tensors in Tucker format

A sequential multilinear Nyström algorithm for streaming low-rank approximation of tensors in Tucker format

An Efficient Randomized Fixed-Precision Algorithm for Tensor Singular Value Decomposition