Abstract:The utilization of sketching techniques has progressively emerged as a pivotal method for enhancing the efficiency of online learning. In linear bandit settings, current sketch-based approaches leverage matrix sketching to reduce the per-round time complexity from \(\Omega\left(d^2\right)\) to \(O(d)\), where \(d\) is the input dimension. Despite this improved efficiency, these approaches encounter critical pitfalls: if the spectral tail of the covariance matrix does not decrease rapidly, it can lead to linear regret. In this paper, we revisit the regret analysis and algorithm design concerning approximating the covariance matrix using matrix sketching in linear bandits. We illustrate how inappropriate sketch sizes can result in unbounded spectral loss, thereby causing linear regret. To prevent this issue, we propose Dyadic Block Sketching, an innovative streaming matrix sketching approach that adaptively manages sketch size to constrain global spectral loss. This approach effectively tracks the best rank-\( k \) approximation in an online manner, ensuring efficiency when the geometry of the covariance matrix is favorable. Then, we apply the proposed Dyadic Block Sketching to linear bandits and demonstrate that the resulting bandit algorithm can achieve sublinear regret without prior knowledge of the covariance matrix, even under the worst case. Our method is a general framework for efficient sketch-based linear bandits, applicable to all existing sketch-based approaches, and offers improved regret bounds accordingly. Additionally, we conduct comprehensive empirical studies using both synthetic and real-world data to validate the accuracy of our theoretical findings and to highlight the effectiveness of our algorithm.

Co-Occuring Directions Sketching for Approximate Matrix Multiply

Frequent Direction Algorithms for Approximate Matrix Multiplication with Applications in CCA.

Near Optimal Frequent Directions for Sketching Dense and Sparse Matrices

Approximate Multiplication of Sparse Matrices with Limited Space

Off-Grid DOA Estimation Algorithm Based on Expanded Matrix and Matrix Pencil

Seeing the Forest from the Trees in Two Looks: Matrix Sketching by Cascaded Bilateral Sampling

Optimal Matrix Sketching over Sliding Windows

Sketching Algorithms for Sparse Dictionary Learning: PTAS and Turnstile Streaming

Localized sketching for matrix multiplication and ridge regression

Sampling Methods for Inner Product Sketching

Efficient Matrix Sketching over Distributed Data

Matrix Sketching in Bandits: Current Pitfalls and New Framework

On Sketching Quadratic Forms

Efficient Bounds and Estimates for Canonical Angles in Randomized Subspace Approximations

Distributed Least Squares in Small Space via Sketching and Bias Reduction

Communication-Efficient Distributed Covariance Sketch, with Application to Distributed PCA

Tensor-Based Sketching Method for the Low-Rank Approximation of Data Streams

Deterministic Clustering in High Dimensional Spaces: Sketches and Approximation

Randomization or Condensation?: Linear-Cost Matrix Sketching Via Cascaded Compression Sampling.