Abstract:Principal Component Analysis (PCA) is a widely used technique in machine learning, data analysis and signal processing. With the increase in the size and complexity of datasets, it has become important to develop low-space usage algorithms for PCA. Streaming PCA has gained significant attention in recent years, as it can handle large datasets efficiently. The kernel method, which is commonly used in learning algorithms such as Support Vector Machines (SVMs), has also been applied in PCA algorithms. We propose a streaming algorithm for Kernel PCA problems based on the traditional scheme by Oja. Our algorithm addresses the challenge of reducing the memory usage of PCA while maintaining its accuracy. We analyze the performance of our algorithm by studying the conditions under which it succeeds. Specifically, we show that, when the spectral ratio $R := \lambda_1/\lambda_2$ of the target covariance matrix is lower bounded by $C \cdot \log n\cdot \log d$, the streaming PCA can be solved with $O(d)$ space cost. Our proposed algorithm has several advantages over existing methods. First, it is a streaming algorithm that can handle large datasets efficiently. Second, it employs the kernel method, which allows it to capture complex nonlinear relationships among data points. Third, it has a low-space usage, making it suitable for applications where memory is limited.

Computation and Store Space Constrained-Based Sparse Kernel Data Analysis

Adaptive Sparse Kernel Principal Component Analysis for Computation and Store Space Constrained-based Feature Extraction.

Sparse Data-Dependent Kernel Principal Component Analysis Based on Least Squares Support Vector Machine for Feature Extraction and Recognition.

Sparse kernel principal component algorithm based on an approximate zero norm

Incomplete Cholesky Decomposition Based Kernel Principal Component Analysis For Large-Scale Data Set

Matrix-based Kernel Principal Component Analysis for Large-Scale Data Set.

Streaming Kernel PCA Algorithm With Small Space

Adaptive kernel subspace method for speeding up feature extraction

Sparse Kernel Feature Extraction Via Support Vector Learning.

KPCA Based on Feature Samples for Fault Detection

To Solve Kernel Principal Component Analysis Using Iterative Method

Sparse Kernel Extreme Components Analysis

An Iterative Algorithm for Robust Kernel Principal Component Analysis

A Framework of Sparse Kernel Principal Component Analysis and Its Applications

Kernel sparse concept coding algorithm and its application for image representation

An Improved Algorithm for Kernel Principal Component Analysis

Sparse Principal Component Analysis With Preserved Sparsity Pattern

Sparse representation image classification algorithm based on principal component analysis convolution

Improved method of dynamic kernel principal component analysis based on feature subspace

Stochastic Optimization for Kernel Pca

Fast algorithm about KPCA