Abstract:Dimensionality reduction, cluster analysis, and sparse representation are basic components in machine learning. However, their relationships have not yet been fully investigated. In this paper, we find that the spectral graph theory underlies a series of these elementary methods and can unify them into a complete framework. The methods include PCA, K-means, Laplacian eigenmap (LE), ratio cut (Rcut), and a new sparse representation method developed by us, called spectral sparse representation (SSR). Further, extended relations to conventional over-complete sparse representations (e.g., method of optimal directions, KSVD), manifold learning (e.g., kernel PCA, multidimensional scaling, Isomap, locally linear embedding), and subspace clustering (e.g., sparse subspace clustering, low-rank representation) are incorporated. We show that, under an ideal condition from the spectral graph theory, PCA, K-means, LE, and Rcut are unified together. And when the condition is relaxed, the unification evolves to SSR, which lies in the intermediate between PCA/LE and K-mean/Rcut. An efficient algorithm, NSCrt, is developed to solve the sparse codes of SSR. SSR combines merits of both sides: its sparse codes reduce dimensionality of data meanwhile revealing cluster structure. For its inherent relation to cluster analysis, the codes of SSR can be directly used for clustering. Scut, a clustering approach derived from SSR reaches the state-of-the-art performance in the spectral clustering family. The one-shot solution obtained by Scut is comparable to the optimal result of K-means that are run many times. Experiments on various data sets demonstrate the properties and strengths of SSR, NSCrt, and Scut.

RESKM: A General Framework to Accelerate Large-Scale Spectral Clustering

A Unified Framework for Representation-Based Subspace Clustering of Out-of-Sample and Large-Scale Data.

Multi-kernel Graph Fusion for Spectral Clustering

Discrete and Balanced Spectral Clustering with Scalability

Scalable Spectral Clustering Using Random Binning Features

Unified Spectral Clustering with Optimal Graph

Spectral clustering with linear embedding: A discrete clustering method for large-scale data

Fast Large-Scale Spectral Clustering By Sequential Shrinkage Optimization

Structured Graph Reconstruction for Scalable Clustering

Large Scale Spectral Clustering Via Landmark-Based Sparse Representation

A new Kmeans clustering model and its generalization achieved by joint spectral embedding and rotation

Scalable Constrained Spectral Clustering

Fast Spectral Clustering Learning with Hierarchical Bipartite Graph for Large-Scale Data

A Restarted Large-Scale Spectral Clustering with Self-Guiding and Block Diagonal Representation

Spectral Sparse Representation for Clustering: Evolved from PCA, K-means, Laplacian Eigenmap, and Ratio Cut

Spectral Embedded Clustering: A Framework for In-Sample and Out-of-Sample Spectral Clustering

Spectral Clustering on Multiple Manifolds

A Convex Formulation for Spectral Shrunk Clustering

Three-Stage Subspace Clustering Framework with Graph-Based Transformation and Optimization

Rank-Constrained Spectral Clustering with Flexible Embedding

Fast spectral clustering of data using sequential matrix compression