Abstract:In this paper, we propose to unify various dimensionality reduction algorithms by interpreting the Manifold Regularization (MR) framework in a new way. Although the MR framework was originally proposed for learning, we utilize it to give a unified treatment for many dimensionality reduction algorithms from linear to nonlinear, supervised to unsupervised, and single class to multi-class approaches. In addition, the framework can provide a general platform to design new dimensionality reduction algorithms. The framework is expressed in the form of a regularized fitting problem in a Reproducing Kernel Hilbert Space. It consists of one error part and two regularization terms: the complexity term and the smoothness term. The error part measures the difference between the estimated (low-dimensional) data distribution and the true (high-dimensional) data distribution or the difference between the estimated and targeted low-dimensional representations of data, the complexity term is a measurement of the complexity of the feature mapping for dimensionality reduction, and the smoothness term reflects the intrinsic structure of data. Based on the framework, we propose a Manifold Regularized Kernel Least Squares (MR-KLS) method which can efficiently learn an explicit feature mapping (in the semi-supervised sense). Experiments show that our approach is effective for out-of-sample extrapolation.

Multiple Kernel Dimensionality Reduction Via Ratio-Trace and Marginal Fisher Analysis

Multiple kernel dimensionality reduction via spectral regression and trace ratio maximization

Generalized Convexity-Based Inexact Projection Method for Multiple Kernel Learning

Spectral Regression Based Marginal Fisher Analysis Dimensionality Reduction Algorithm.

Multiple Kernel Spectral Regression for Dimensionality Reduction

Efficient Multiple Kernel Learning Algorithms Using Low-Rank Representation.

Dimensionality Reduction by Mixed Kernel Canonical Correlation Analysis

A General Framework for Dimensionality Reduction of K-Means Clustering

Face Recognition Algorithm Based on Adaptive Kernel Marginal Fisher Analysis

Multiple Kernel Learning via Low-Rank Nonnegative Matrix Factorization for Classification of Hyperspectral Imagery

Multiple Kernel Learning In Fisher Discriminant Analysis For Face Recognition

Marginal Fisher Analysis with Polynomial Matrix Function

Unsupervised Multiple Kernel Learning

Manifold regularization Multiple Kernel Learning machine for classification

Max-K-Min Distance Analysis for Dimension Reduction

Supervised Dimensionality Reduction Technology of Generalized Discriminant Component Analysis and Its Kernelization Forms

Joint Learning of Distance Metric and Kernel Classifier via Multiple Kernel Learning.

Marginal Fisher Analysis and Its Variants for Human Gait Recognition and Content- Based Image Retrieval

Dimensionality Reduction: an Interpretation from Manifold Regularization Perspective.

Regularized Embedded Multiple Kernel Dimensionality Reduction for Mine Signal Processing.

Nonparametric Marginal Fisher Analysis for Feature Extraction