Abstract:We prove, using the subspace embedding guarantee in a black box way, that one can achieve the spectral norm guarantee for approximate matrix multiplication with a dimensionality-reducing map having $m = O(\tilde{r}/\varepsilon^2)$ rows. Here $\tilde{r}$ is the maximum stable rank, i.e. squared ratio of Frobenius and operator norms, of the two matrices being multiplied. This is a quantitative improvement over previous work of [MZ11, KVZ14], and is also optimal for any oblivious dimensionality-reducing map. Furthermore, due to the black box reliance on the subspace embedding property in our proofs, our theorem can be applied to a much more general class of sketching matrices than what was known before, in addition to achieving better bounds. For example, one can apply our theorem to efficient subspace embeddings such as the Subsampled Randomized Hadamard Transform or sparse subspace embeddings, or even with subspace embedding constructions that may be developed in the future. Our main theorem, via connections with spectral error matrix multiplication shown in prior work, implies quantitative improvements for approximate least squares regression and low rank approximation. Our main result has also already been applied to improve dimensionality reduction guarantees for $k$-means clustering [CEMMP14], and implies new results for nonparametric regression [YPW15]. We also separately point out that the proof of the "BSS" deterministic row-sampling result of [BSS12] can be modified to show that for any matrices $A, B$ of stable rank at most $\tilde{r}$, one can achieve the spectral norm guarantee for approximate matrix multiplication of $A^T B$ by deterministically sampling $O(\tilde{r}/\varepsilon^2)$ rows that can be found in polynomial time. The original result of [BSS12] was for rank instead of stable rank. Our observation leads to a stronger version of a main theorem of [KMST10].

Norm and Trace Estimation with Random Rank-one Vectors

Fast Randomized Numerical Rank Estimation for Numerically Low-Rank Matrices

On randomized trace estimates for indefinite matrices with an application to determinants

Rank Is All You Need: Estimating the Trace of Powers of Density Matrices

XTrace: Making the Most of Every Sample in Stochastic Trace Estimation

Computation of the von Neumann entropy of large matrices via trace estimators and rational Krylov methods

On The Variance of Schatten $p$-Norm Estimation with Gaussian Sketching Matrices

Improved bounds for randomized Schatten norm estimation of numerically low-rank matrices

The rank of a random triangular matrix over $\mathbb{F}_q$

Estimation of high-dimensional low-rank matrices

On the Unreasonable Effectiveness of Single Vector Krylov Methods for Low-Rank Approximation

Optimal approximate matrix product in terms of stable rank

Randomized Nyström approximation of non-negative self-adjoint operators

Tractable and Scalable Schatten Quasi-Norm Approximations for Rank Minimization

Restricted Isometry Property of Rank-One Measurements with Random Unit-Modulus Vectors

A randomized algorithm to solve reduced rank operator regression

Von Neumann Entropy Penalization and Low Rank Matrix Estimation

Rank-Constrained Least-Squares: Prediction and Inference

Distributed Estimation of Generalized Matrix Rank: Efficient Algorithms and Lower Bounds