Abstract:This paper addresses the scalability issue in spectral analysis which has been widely used in data management applications. Spectral analysis techniques enjoy powerful clustering capability while suffer from high computational complexity. In most of previous research, the bottleneck of computational complexity of spectral analysis stems from the construction of pairwise similarity matrix among objects, which costs at least O(n2) where n is the number of the data points. In this paper, we propose a novel estimator of the similarity matrix using K-means accumulative consensus matrix which is intrinsically sparse. The computational cost of the accumulative consensus matrix is O(nlogn). We further develop a Non-negative Matrix Factorization approach to derive clustering assignment. The overall complexity of our approach remains O(nlogn). In order to validate our method, we (1) theoretically show the local preserving and convergent property of the similarity estimator, (2) validate it by a large number of real world datasets and compare the results to other state-of-the-art spectral analysis, and (3) apply it to large-scale data clustering problems. Results show that our approach uses much less computational time than other state-of-the-art clustering methods, meanwhile provides comparable clustering qualities. We also successfully apply our approach to a 5-million dataset on a single machine using reasonable time. Our techniques open a new direction for high-quality large-scale data analysis.

Spectral clustering for time series

Spectral clustering method based on independent component analysis for time series

Spectral Clustering And Long Timeseries Classification

Spectral Analysis of High-dimensional Time Series

Time Series Clustering via Inter-network Heterogeneity

Spectral clustering algorithm based on adaptive neighbor distance sort order

Consensus Spectral Clustering in Near-Linear Time

Semi-supervised Spectral Clustering of Time-Series Similarity

Spectral Preprocessing for Clustering Time-Series Gene Expressions

A Self-Adaptive Spectral Clustering Algorithm

Spectral clustering with variance information for group structure estimation in panel data

Spectral clustering of time-evolving networks using the inflated dynamic Laplacian for graphs

Spectral Embedded Adaptive Neighbors Clustering.

A spectral clustering algorithm for automatically determining clusters number

A Genetic Spectral Clustering Algorithm

A Novel Spectral Clustering Algorithm Based on Neighbor Relation and Gaussian Kernel Function with Only One Parameter

Streaming spectral clustering

Spectral clustering of graphs

Spectral Aggregation for Clustering Ensemble

Affinity Learning Via Self-Supervised Diffusion for Spectral Clustering

Research of Spectral Clustering Based on Graph Partition