Abstract:Kernel trick has achieved remarkable success in various machine learning tasks, especially those with high-dimensional non-linear data. In addition, these data usually tend to have compact representation that cluster in a low-dimensional subspace. In order to offer a general and comprehensive framework for high-dimensional non-linear data, in this paper, we generalizes multiple kernel learning and subspace learning in a reconstructed reproducing kernel Hilbert space (RKHS) endowed with manifold leaning. First, we construct reconstructed kernels by fusing manifold learning and some base kernel functions, and then learn the optimal kernel by linearly combining the reconstructed kernels. The proposed MKL method can introduce different prior knowledge such as neighborhood information and classification information, to solve different tasks of high-dimensional data. Furthermore, we propose a subspace learning based on RKHS reconstruction, named MVSL for short, of which the objective function is designed with variance maximization criterion, and use an iterative algorithm to solve it. We also incorporates data discriminant information to the learning process of the modified kernel by kernel alignment criterion and a regularization term, to learning the optimal kernel matrix for RKHS reconstruction, and propose another subspace learning method, named Discriminative MVSL. Experimental results on toy and real-world datasets demonstrate that the proposed MKL and subspace learning methods are able to learn the local manifold and the global statistics information of data based on RKHS reconstruction, and thus they achieve a satisfactory performance on classification and dimension reduction tasks.

IKLTSA: An Incremental Kernel LTSA Method

Incremental manifold learning via tangent space alignment

Incremental Laplacian Eigenmaps by Preserving Adjacent Information Between Data Points

A New Manifold Learning Algorithm Based on Incremental Spectral Decomposition.

An Incremental Manifold Learning Algorithm Based on Iterative Decomposition

A Manifold Learning Algorithm Based on Incremental Tangent Space Alignment

Improving Classification Precision by Implicit Kernels Motivated by Manifold Learning

Independent Component Analysis: Embedded LTSA

Incremental Alignment Manifold Learning

RKHS reconstruction based on manifold learning for high-dimensional data

A Kernel PCA View of the Local Tangent Space Alignment Algorithm

LKLR: A Local Tangent Space-Alignment Kernel Least-Squares Regression Algorithm

Nonlinear Dimensionality Reduction by Locally Linear Inlaying

Nonlinear Process Monitoring Using Improved Kernel Principal Component Analysis

Kernel Based Incremental Learning Isomap Algorithm

Learning Image Manifold Via Local Tensor Subspace Alignment

Incremental Manifold Learning Algorithm Using PCA on Overlapping Local Neighborhoods for Dimensionality Reduction

Kernel-Induced Space Selection Approach to LPKHDA Dimensional Reduction Algorithm

Incremental Manifold Learning by Spectral Embedding Methods

Incremental and evolutionary manifold learning: a survey

Manifold Regularization for Kernelized LSTD