Abstract:Matrix sketching is aimed at finding close approximations of a matrix by factors of much smaller dimensions, which has important applications in optimization and machine learning. Given a matrix A of size m by n, state-of-the-art randomized algorithms take O(m * n) time and space to obtain its low-rank decomposition. Although quite useful, the need to store or manipulate the entire matrix makes it a computational bottleneck for truly large and dense inputs. Can we sketch an m-by-n matrix in O(m + n) cost by accessing only a small fraction of its rows and columns, without knowing anything about the remaining data? In this paper, we propose the cascaded bilateral sampling (CABS) framework to solve this problem. We start from demonstrating how the approximation quality of bilateral matrix sketching depends on the encoding powers of sampling. In particular, the sampled rows and columns should correspond to the code-vectors in the ground truth decompositions. Motivated by this analysis, we propose to first generate a pilot-sketch using simple random sampling, and then pursue more advanced, "follow-up" sampling on the pilot-sketch factors seeking maximal encoding powers. In this cascading process, the rise of approximation quality is shown to be lower-bounded by the improvement of encoding powers in the follow-up sampling step, thus theoretically guarantees the algorithmic boosting property. Computationally, our framework only takes linear time and space, and at the same time its performance rivals the quality of state-of-the-art algorithms consuming a quadratic amount of resources. Empirical evaluations on benchmark data fully demonstrate the potential of our methods in large scale matrix sketching and related areas.

Tensor-Based Sketching Method for the Low-Rank Approximation of Data Streams

Effective Streaming Low-Tubal-Rank Tensor Approximation via Frequent Directions

Practical Sketching Algorithms for Low-Rank Tucker Approximation of Large Tensors

Low-Rank Tucker Approximation of a Tensor from Streaming Data

Streaming Anomaly Detection Using Randomized Matrix Sketching

Tensor-structured sketching for constrained least squares

Sketching Algorithms for Sparse Dictionary Learning: PTAS and Turnstile Streaming

Low-rank Tensor Train Decomposition Using TensorSketch

Robust recovery of low-rank matrices and low-tubal-rank tensors from noisy sketches

Randomized algorithms for streaming low-rank approximation in tree tensor network format

Sketching Linear Classifiers over Data Streams

High-Dimensional Geometric Streaming for Nearly Low Rank Data

OneSketch: A Generic and Accurate Sketch for Data Streams

Subspace Learning and Imputation for Streaming Big Data Matrices and Tensors

On-Line Network Traffic Anomaly Detection Based on Tensor Sketch

Seeing the Forest from the Trees in Two Looks: Matrix Sketching by Cascaded Bilateral Sampling

Learning-Based Low-Rank Approximations

Tracking Tensor Ring Decompositions of Streaming Tensors

Fast Sampling Based Sketches for Tensors

Fast Rotation Kernel Density Estimation over Data Streams

A sequential multilinear Nyström algorithm for streaming low-rank approximation of tensors in Tucker format