Abstract:Principal Component Analysis (PCA) and its nonlinear extension Kernel PCA (KPCA) are widely used across science and industry for data analysis and dimensionality reduction. Modern deep learning tools have achieved great empirical success, but a framework for deep principal component analysis is still lacking. Here we develop a deep kernel PCA methodology (DKPCA) to extract multiple levels of the most informative components of the data. Our scheme can effectively identify new hierarchical variables, called deep principal components, capturing the main characteristics of high-dimensional data through a simple and interpretable numerical optimization. We couple the principal components of multiple KPCA levels, theoretically showing that DKPCA creates both forward and backward dependency across levels, which has not been explored in kernel methods and yet is crucial to extract more informative features. Various experimental evaluations on multiple data types show that DKPCA finds more efficient and disentangled representations with higher explained variance in fewer principal components, compared to the shallow KPCA. We demonstrate that our method allows for effective hierarchical data exploration, with the ability to separate the key generative factors of the input data both for large datasets and when few training samples are available. Overall, DKPCA can facilitate the extraction of useful patterns from high-dimensional data by learning more informative features organized in different levels, giving diversified aspects to explore the variation factors in the data, while maintaining a simple mathematical formulation.

Improvement of variables interpretability in kernel PCA

The Optimization for Parameter of Kernel Function with GA

Adaptive Kpca Modeling of Nonlinear Systems

Cross-validatory Framework for Optimal Parameter Estimation of KPCA and KPLS Models

To Solve Kernel Principal Component Analysis Using Iterative Method

Nonlinear Process Monitoring Using Improved Kernel Principal Component Analysis

Improved kernel PCA-based monitoring approach for nonlinear processes

An Iterative Algorithm for Robust Kernel Principal Component Analysis

Learning a Data-Dependent Kernel Function for KPCA-based Nonlinear Process Monitoring

A General Kernelization Framework for Learning Algorithms Based on Kernel PCA.

Statistical Optimality and Computational Efficiency of Nyström Kernel PCA

Generalisation bounds for kernel PCA through PAC‐Bayes learning

Adaptive kernel subspace method for speeding up feature extraction

Incomplete Cholesky Decomposition Based Kernel Principal Component Analysis For Large-Scale Data Set

Invertible Kernel PCA with Random Fourier Features

Deep Kernel Principal Component Analysis for Multi-level Feature Learning

Multi-kernel PCA based high-dimensional images feature reduction

Learning with Cross-Kernels and Ideal PCA

A Framework of Sparse Kernel Principal Component Analysis and Its Applications

Robust and Sparse Kernel PCA and Its Outlier Map

An Infinite Dimensional Analysis of Kernel Principal Components