Abstract:The statistical analysis of Randomized Numerical Linear Algebra (RandNLA) algorithms within the past few years has mostly focused on their performance as point estimators. However, this is insufficient for conducting statistical inference, e.g., constructing confidence intervals and hypothesis testing, since the distribution of the estimator is lacking. In this article, we develop an asymptotic analysis to derive the distribution of RandNLA sampling estimators for the least-squares problem. In particular, we derive the asymptotic distribution of a general sampling estimator with arbitrary sampling probabilities. The analysis is conducted in two complementary settings, i.e., when the objective of interest is to approximate the full sample estimator or is to infer the underlying ground truth model parameters. For each setting, we show that the sampling estimator is asymptotically normally distributed under mild regularity conditions. Moreover, the sampling estimator is asymptotically unbiased in both settings. Based on our asymptotic analysis, we use two criteria, the Asymptotic Mean Squared Error (AMSE) and the Expected Asymptotic Mean Squared Error (EAMSE), to identify optimal sampling probabilities. Several of these optimal sampling probability distributions are new to the literature, e.g., the root leverage sampling estimator and the predictor length sampling estimator. Our theoretical results clarify the role of leverage in the sampling process, and our empirical results demonstrate improvements over existing methods.

Randomized numerical linear algebra: Foundations and algorithms

Randomized algorithms for low-rank matrix approximation: Design, analysis, and applications

Fast and Accurate Randomized Algorithms for Linear Systems and Eigenvalue Problems

Recent and Upcoming Developments in Randomized Numerical Linear Algebra for Machine Learning

Randomized Iterative Methods for Linear Systems

Randomized matrix computations: Themes and variations

Asymptotic Analysis of Sampling Estimators for Randomized Numerical Linear Algebra Algorithms

On pseudoinverse-free randomized methods for linear systems: Unified framework and acceleration

Johnson-Lindenstrauss Lemma, Linear and Nonlinear Random Projections, Random Fourier Features, and Random Kitchen Sinks: Tutorial and Survey

Randomized algorithms for distributed computation of principal component analysis and singular value decomposition

Determinantal Point Processes in Randomized Numerical Linear Algebra

Randomized Methods for Linear Constraints: Convergence Rates and Conditioning

Randomised subspace methods for non-convex optimization, with applications to nonlinear least-squares

Pseudoinverse-free randomized block iterative algorithms for consistent and inconsistent linear systems

A literature survey of matrix methods for data science

Randomized algorithms for low-rank matrix factorizations: sharp performance bounds

Iterative methods for solving factorized linear systems

Randomized Block Adaptive Linear System Solvers

A Note on the Randomized Kaczmarz Algorithm for Solving Doubly Noisy Linear Systems

Randomized sketched TT-GMRES for linear systems with tensor structure