Abstract:The Johnson--Lindenstrauss (JL) lemma is a powerful tool for dimensionality reduction in modern algorithm design. The lemma states that any set of high-dimensional points in a Euclidean space can be flattened to lower dimensions while approximately preserving pairwise Euclidean distances. Random matrices satisfying this lemma are called JL transforms (JLTs). Inspired by existing $s$-hashing JLTs with exactly $s$ nonzero elements on each column, the present work introduces an ensemble of sparse matrices encompassing so-called $s$-hashing-like matrices whose expected number of nonzero elements on each column is~$s$. The independence of the sub-Gaussian entries of these matrices and the knowledge of their exact distribution play an important role in their analyses. Using properties of independent sub-Gaussian random variables, these matrices are demonstrated to be JLTs, and their smallest and largest singular values are estimated non-asymptotically using a technique from geometric functional analysis. As the dimensions of the matrix grow to infinity, these singular values are proved to converge almost surely to fixed quantities (by using the universal Bai--Yin law), and in distribution to the Gaussian orthogonal ensemble (GOE) Tracy--Widom law after proper rescalings. Understanding the behaviors of extreme singular values is important in general because they are often used to define a measure of stability of matrix algorithms. For example, JLTs were recently used in derivative-free optimization algorithmic frameworks to select random subspaces in which are constructed random models or poll directions to achieve scalability, whence estimating their smallest singular value in particular helps determine the dimension of these subspaces.

On Singular Values Distribution of A Matrix Large Auto-Covariance in the Ultra-Dimensional Regime

Moment Approach for Singular Values Distribution of a Large Auto-Covariance Matrix

On asymptotics of eigenvectors of large sample covariance matrix

Extreme value analysis for the sample autocovariance matrices of heavy-tailed multivariate time series

Convergence Rates of Spectral Distributions of Large Sample Covariance Matrices

A Note on the Convergence Rate of the Spectral Distributions of Large Random Matrices

On limiting spectral distribution of large sample covariance matrices by VARMA(p,q)

A Note on the Convergence Rate of the Spectral Distributions of Large Sample Covariance Matrices

On Eigenvalues of a High-Dimensional Spatial-Sign Covariance Matrix

A necessary and sufficient condition for edge universality at the largest singular values of covariance matrices

Large N Limit for the Product of Two Coupled Random Matrices

On limit theorem for the eigenvalues of product of two random matrices

Exact Separation of Eigenvalues of Large Dimensional Noncentral Sample Covariance Matrices

Analysis of the limiting spectral distribution of large dimensional General information-plus-noise type matrices

Universality of a double scaling limit near singular edge points in random matrix models

A Note on the Limiting Mean Distribution of Singular Values for Products of Two Wishart Random Matrices

Almost sure convergence of the largest and smallest eigenvalues of high-dimensional sample correlation matrices

A class of sparse Johnson--Lindenstrauss transforms and analysis of their extreme singular values

Extreme Eigenvalues of Large-Dimensional Spiked Fisher Matrices with Application

The Asymptotic Properties of the Extreme Eigenvectors of High-dimensional Generalized Spiked Covariance Model

Non-asymptotic theory of random matrices: extreme singular values