Abstract:Dimensionality reduction and feature subset selection are two techniques for reducing the attribute space of a feature set, which is an important component of both supervised and unsupervised classification or regression problems. While in feature subset selection a subset of the original attributes is extracted, dimensionality reduction in general produces linear combinations of the original attribute set. In this paper we investigate the relationship between several attribute space reduction techniques and the resulting classification accuracy for two very different application areas. On the one hand, we consider e-mail filtering, where the feature space contains various properties of e-mail messages, and on the other hand, we consider drug discovery problems, where quantitative representations of molecular structures are encoded in terms of information-preserving descriptor values. Subsets of the original attributes constructed by filter and wrapper techniques as well as subsets of linear combinations of the original attributes constructed by three different variants of the principle component analysis (PCA) are compared in terms of the classification performance achieved with various machine learning algorithms as well as in terms of runtime performance. We successively reduce the size of the attribute sets and investigate the changes in the classification results. Moreover, we explore the relationship between the variance captured in the linear combinations within PCA and the resulting classification accuracy. The results show that the classification accuracy based on PCA is highly sensitive to the type of data and that the variance captured the principal components is not necessarily a vital indicator for the classification performance.

Dimensionality Reduction and Pattern Classification Method Based on the SVM and PCA

Computational and Theoretical Analysis of Supervised Dimensionality Reduction

Multivariate time series classification based on μσ-DWC feature and tree-structured M-SVM

SVM Classification:Its Contents and Challenges

Face Recognition Based on Partial Least Squares and Support Vector Machine

Image classification base on PCA of multi-view deep representation

An Adaptive Class Pairwise Dimensionality Reduction Algorithm

Image Classification Based on KPCA and SVM with Randomized Hyper-parameter Optimization

A Brief Survey of Dimension Reduction

A Comparison of Classification Accuracy Achieved with Wrappers, Filters and PCA

Supervised Linear Dimension-Reduction Methods: Review, Extensions, and Comparisons

Impacts of Feature Dimensionality to the Support Vector Machine Classifier for Hyperspectral Remote Sensing Image

A Comparison Study on Nonlinear Dimension Reduction Methods with Kernel Variations: Visualization, Optimization and Classification.

SVM Modeling and Application Based on Kernel Function Principal Component Analysis

Y-SPCR: A New Dimensionality Reduction Method for Gene Expression Data Classification

Discuss and Research of Face Recognition Based on PCA Algorithm

On the Relationship Between Feature Selection and Classification Accuracy

PCA-KL: a parametric dimensionality reduction approach for unsupervised metric learning

Optimization Algorithm with Kernel PCA to Support Vector Machines for Time Series Prediction

Comparative Analysis of 2D-PCA Based Dimensionality Reduction and Feature Extraction

The Application in Classification among Fisher Discriminant Criterion K-L Conversion and SVM