Abstract:Spectral methods have emerged as a simple yet surprisingly effective approach for extracting information from massive, noisy and incomplete data. In a nutshell, spectral methods refer to a collection of algorithms built upon the eigenvalues (resp. singular values) and eigenvectors (resp. singular vectors) of some properly designed matrices constructed from data. A diverse array of applications have been found in machine learning, imaging science, financial and econometric modeling, and signal processing, including recommendation systems, community detection, ranking, structured matrix recovery, tensor data estimation, joint shape matching, blind deconvolution, financial investments, risk managements, treatment evaluations, causal inference, amongst others. Due to their simplicity and effectiveness, spectral methods are not only used as a stand-alone estimator, but also frequently employed to facilitate other more sophisticated algorithms to enhance performance. While the studies of spectral methods can be traced back to classical matrix perturbation theory and the method of moments, the past decade has witnessed tremendous theoretical advances in demystifying their efficacy through the lens of statistical modeling, with the aid of concentration inequalities and non-asymptotic random matrix theory. This monograph aims to present a systematic, comprehensive, yet accessible introduction to spectral methods from a modern statistical perspective, highlighting their algorithmic implications in diverse large-scale applications. In particular, our exposition gravitates around several central questions that span various applications: how to characterize the sample efficiency of spectral methods in reaching a target level of statistical accuracy, and how to assess their stability in the face of random noise, missing data, and adversarial corruptions? In addition to conventional ℓ2 perturbation analysis, we present a systematic ℓ∞ and ℓ2,∞ perturbation theory for eigenspace and singular subspaces, which has only recently become available owing to a powerful “leave-one-out” analysis framework.

Unit-Sparse Data and Covariance Model Fitting: Discrete Fourier Transformation and Minimum Variance Distortless Response

Parameter Estimation in Exponential Models by Linear and Nonlinear Fitting Methods.

A Regularized High-Dimensional Positive Definite Covariance Estimator with High-Frequency Data

A Variance Minimization Criterion to Feature Selection Using Laplacian Regularization

Large-Dimensional Positive Definite Covariance Estimation for High Frequency Data via Low-rank and Sparse Matrix Decomposition

Sparse Spatial Spectral Fitting with Nonuniform Noise Covariance Matrix Estimation Based on Semidefinite Optimization

Fast Covariance Estimation for Sparse Functional Data

Spectral Methods for Data Science: A Statistical Perspective

Fundamentals of Ecological Modelling, 3rd Edition

Minimum Covariance Determinant: Spectral Embedding and Subset Size Determination

Theory of functional principal component analysis for discretely observed data

Variance constrained partial least squares

A Geometric Unification of Distributionally Robust Covariance Estimators: Shrinking the Spectrum by Inflating the Ambiguity Set

Prediction Weighted Maximum Frequency Selection

Adaptive filtering under minimum information divergence criterion

Fast covariance estimation for multivariate sparse functional data

Covariance Model with General Linear Structure and Divergent Parameters

A Kalman-Filter-Incorporated Latent Factor Analysis Model for Temporally Dynamic Sparse Data

Nonparametric Operator-Regularized Covariance Function Estimation for Functional Data

A New Linear Regression Kalman Filter with Symmetric Samples.

Model Misfit Minimization