Abstract:Spectral methods have emerged as a simple yet surprisingly effective approach for extracting information from massive, noisy and incomplete data. In a nutshell, spectral methods refer to a collection of algorithms built upon the eigenvalues (resp. singular values) and eigenvectors (resp. singular vectors) of some properly designed matrices constructed from data. A diverse array of applications have been found in machine learning, imaging science, financial and econometric modeling, and signal processing, including recommendation systems, community detection, ranking, structured matrix recovery, tensor data estimation, joint shape matching, blind deconvolution, financial investments, risk managements, treatment evaluations, causal inference, amongst others. Due to their simplicity and effectiveness, spectral methods are not only used as a stand-alone estimator, but also frequently employed to facilitate other more sophisticated algorithms to enhance performance. While the studies of spectral methods can be traced back to classical matrix perturbation theory and the method of moments, the past decade has witnessed tremendous theoretical advances in demystifying their efficacy through the lens of statistical modeling, with the aid of concentration inequalities and non-asymptotic random matrix theory. This monograph aims to present a systematic, comprehensive, yet accessible introduction to spectral methods from a modern statistical perspective, highlighting their algorithmic implications in diverse large-scale applications. In particular, our exposition gravitates around several central questions that span various applications: how to characterize the sample efficiency of spectral methods in reaching a target level of statistical accuracy, and how to assess their stability in the face of random noise, missing data, and adversarial corruptions? In addition to conventional ℓ2 perturbation analysis, we present a systematic ℓ∞ and ℓ2,∞ perturbation theory for eigenspace and singular subspaces, which has only recently become available owing to a powerful “leave-one-out” analysis framework.

Towards Large Scale Spectral Problems via Diffusion Process

Large Scale Spectral Clustering Via Landmark-Based Sparse Representation

Large Graph Clustering with Simultaneous Spectral Embedding and Discretization

Spectral multidimensional scaling

Large Scale Spectral Clustering Based On Fast Landmark Sampling

Diffusion Process with Structural Changes for Subspace Clustering

Making Large-Scale Nyström Approximation Possible

Spectral clustering based on iterative optimization for large-scale and high-dimensional data.

Spectral Algorithms on Manifolds through Diffusion

Computation of Spectral Clustering Algorithm for Large-scale Data Set

Large-Scale Spectral Clustering Based on Representative Points

Spectral clustering with linear embedding: A discrete clustering method for large-scale data

RESKM: A General Framework to Accelerate Large-Scale Spectral Clustering

Spectral Methods for Data Science: A Statistical Perspective

A Unified Framework for Representation-Based Subspace Clustering of Out-of-Sample and Large-Scale Data.

Affinity Learning Via Self-Supervised Diffusion for Spectral Clustering

Spectral graph partitioning based on a random walk diffusion similarity measure

Consensus Spectral Clustering in Near-Linear Time

Fast Optimization of Spectral Embedding and Improved Spectral Rotation

Scalable Spectral Clustering Using Random Binning Features

EFFICIENT SCALING AND MOVING TECHNIQUES FOR SPECTRAL METHODS IN UNBOUNDED DOMAINS