Abstract:Spectral methods have emerged as a simple yet surprisingly effective approach for extracting information from massive, noisy and incomplete data. In a nutshell, spectral methods refer to a collection of algorithms built upon the eigenvalues (resp. singular values) and eigenvectors (resp. singular vectors) of some properly designed matrices constructed from data. A diverse array of applications have been found in machine learning, imaging science, financial and econometric modeling, and signal processing, including recommendation systems, community detection, ranking, structured matrix recovery, tensor data estimation, joint shape matching, blind deconvolution, financial investments, risk managements, treatment evaluations, causal inference, amongst others. Due to their simplicity and effectiveness, spectral methods are not only used as a stand-alone estimator, but also frequently employed to facilitate other more sophisticated algorithms to enhance performance. While the studies of spectral methods can be traced back to classical matrix perturbation theory and the method of moments, the past decade has witnessed tremendous theoretical advances in demystifying their efficacy through the lens of statistical modeling, with the aid of concentration inequalities and non-asymptotic random matrix theory. This monograph aims to present a systematic, comprehensive, yet accessible introduction to spectral methods from a modern statistical perspective, highlighting their algorithmic implications in diverse large-scale applications. In particular, our exposition gravitates around several central questions that span various applications: how to characterize the sample efficiency of spectral methods in reaching a target level of statistical accuracy, and how to assess their stability in the face of random noise, missing data, and adversarial corruptions? In addition to conventional ℓ2 perturbation analysis, we present a systematic ℓ∞ and ℓ2,∞ perturbation theory for eigenspace and singular subspaces, which has only recently become available owing to a powerful “leave-one-out” analysis framework.

Survey on Spectral Machine Learning

Research on Spectral Clustering in Machine Learning

Application Of Partial Least Squares Support Vector Machines (Pls-Svm) In Spectroscopy Quantitative Analysis

Research on Spectral Clustering Algorithms and Prospects

Spectral Methods for Data Science: A Statistical Perspective

Fusion of Manifold Learning and Spectral Clustering Algorithmwith Applications to Fault Diagnosis

Spectral Clustering on Multiple Manifolds

Spectral Methods for Semi-Supervised Manifold Learning.

Dimensionality reduction based on spectral graph and its application

Spectral Learning for Supervised Topic Models

A Survey of Clustering Algorithms Based on Spectra of Graphs

Spectral Neural Networks: Approximation Theory and Optimization Landscape

Machine-learning-based spectral methods for partial differential equations

Lifelong Spectral Clustering.

Spectral clustering with scale fairness constraints

Machine Learning and Deep Learning Techniques for Spectral Spatial Classification of Hyperspectral Images: A Comprehensive Survey

Machine learning based hyperspectral image analysis: A survey

The latest research progress on spectral clustering

A Unified Framework for Discrete Spectral Clustering.

Learning Theory of Distributed Spectral Algorithms

Spectral, Probabilistic, and Deep Metric Learning: Tutorial and Survey