Abstract:Because of high dimensionality, correlation among covariates, and noise contained in data, dimension reduction (DR) techniques are often employed to the application of machine learning algorithms. Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and their kernel variants (KPCA, KLDA) are among the most popular DR methods. Recently, Supervised Kernel Principal Component Analysis (SKPCA) has been shown as another successful alternative. In this paper, brief reviews of these popular techniques are presented first. We then conduct a comparative performance study based on three simulated datasets, after which the performance of the techniques are evaluated through application to a pattern recognition problem in face image analysis. The gender classification problem is considered on MORPH-II and FG-NET, two popular longitudinal face aging databases. Several feature extraction methods are used, including biologically-inspired features (BIF), local binary patterns (LBP), histogram of oriented gradients (HOG), and the Active Appearance Model (AAM). After applications of DR methods, a linear support vector machine (SVM) is deployed with gender classification accuracy rates exceeding 95% on MORPH-II, competitive with benchmark results. A parallel computational approach is also proposed, attaining faster processing speeds and similar recognition rates on MORPH-II. Our computational approach can be applied to practical gender classification systems and generalized to other face analysis tasks, such as race classification and age prediction.

Empirical Evaluation of Kernel PCA Approximation Methods in Classification Tasks

Statistical Optimality and Computational Efficiency of Nyström Kernel PCA

Cross-validatory Framework for Optimal Parameter Estimation of KPCA and KPLS Models

Adaptive Kpca Modeling of Nonlinear Systems

A Comparison Study on Nonlinear Dimension Reduction Methods with Kernel Variations: Visualization, Optimization and Classification.

An Iterative Algorithm for Robust Kernel Principal Component Analysis

Nonlinear Process Monitoring Using Improved Kernel Principal Component Analysis

Tutorial on PCA and approximate PCA and approximate kernel PCA

A Communication-Efficient Distributed Algorithm for Kernel Principal Component Analysis.

Image Classification Based on KPCA and SVM with Randomized Hyper-parameter Optimization

A Framework of Sparse Kernel Principal Component Analysis and Its Applications

Adaptive kernel subspace method for speeding up feature extraction

Generalisation bounds for kernel PCA through PAC‐Bayes learning

Sparse Data-Dependent Kernel Principal Component Analysis Based on Least Squares Support Vector Machine for Feature Extraction and Recognition.

To Solve Kernel Principal Component Analysis Using Iterative Method

Learning a Data-Dependent Kernel Function for KPCA-based Nonlinear Process Monitoring

Ensemble Principal Component Analysis

Incomplete Cholesky Decomposition Based Kernel Principal Component Analysis For Large-Scale Data Set

Hyperspectral Remote Sensing Image Dimensionality Reduction by Kernel PCA Method Based on Nystrm Algorithm

PCA-KL: a parametric dimensionality reduction approach for unsupervised metric learning

Pcaf: Scalable, High Precision K-Nn Search Using Principal Component Analysis Based Filtering