Abstract:Covariance matrices are fundamental to the analysis and forecast of economic, physical and biological systems. Although the eigenvalues $\{\lambda_i\}$ and eigenvectors $\{{\bf u}_i\}$ of a covariance matrix are central to such endeavors, in practice one must inevitably approximate the covariance matrix based on data with finite sample size $n$ to obtain empirical eigenvalues $\{\tilde{\lambda}_i\}$ and eigenvectors $\{\tilde{\bf u}_i\}$, and therefore understanding the error so introduced is of central importance. We analyze eigenvector error $\|{\bf u}_i - \tilde{\bf u}_i \|^2$ while leveraging the assumption that the true covariance matrix having size $p$ is drawn from a matrix ensemble with known spectral properties---particularly, we assume the distribution of population eigenvalues weakly converges as $p\to\infty$ to a spectral density $\rho(\lambda)$ and that the spacing between population eigenvalues is similar to that for the Gaussian orthogonal ensemble. Our approach complements previous analyses of eigenvector error that require the full set of eigenvalues to be known, which can be computationally infeasible when $p$ is large. To provide a scalable approach for uncertainty quantification of eigenvector error, we consider a fixed eigenvalue $\lambda$ and approximate the distribution of the expected square error $r= \mathbb{E}\left[\| {\bf u}_i - \tilde{\bf u}_i \|^2\right]$ across the matrix ensemble for all ${\bf u}_i$ associated with $\lambda_i=\lambda$. We find, for example, that for sufficiently large matrix size $p$ and sample size $n>p$, the probability density of $r$ scales as $1/nr^2$. This power-law scaling implies that eigenvector error is extremely heterogeneous---even if $r$ is very small for most eigenvectors, it can be large for others with non-negligible probability. We support this and further results with numerical experiments.

Nonparametric bootstrap of high-dimensional sample covariance matrices

Cluster-Robust Bootstrap Inference in Quantile Regression Models

A Bootstrap Hypothesis Test for High-Dimensional Mean Vectors

Improved Rates of Bootstrap Approximation for the Operator Norm: A Coordinate-Free Approach

High-dimensional bootstrap and asymptotic expansion

Bootstrapping High Dimensional Time Series

Quantitative limit theorems and bootstrap approximations for empirical spectral projectors

Gaussian and Bootstrap Approximations for Suprema of Empirical Processes

Robust Max Statistics for High-Dimensional Inference

Möbius inversion and the bootstrap

Spectral analysis of high-dimensional sample covariance matrices with missing observations

Testing the number of common factors by bootstrapped sample covariance matrix in high-dimensional factor models

Almost sure convergence of the largest and smallest eigenvalues of high-dimensional sample correlation matrices

Linear spectral statistics of sequential sample covariance matrices

Ensemble-based estimates of eigenvector error for empirical covariance matrices

High-dimensional MANOVA via Bootstrapping and its Application to Functional and Sparse Count Data

Multi-sample hypothesis testing of high-dimensional mean vectors under covariance heterogeneity

Two sample test for covariance matrices in ultra-high dimension

Scalable Nonparametric Sampling from Multimodal Posteriors with the Posterior Bootstrap

Covariance Estimation in High Dimensions Via Kronecker Product Expansions

Randomized incomplete $U$-statistics in high dimensions