Abstract:Dimensionality reduction and feature subset selection are two techniques for reducing the attribute space of a feature set, which is an important component of both supervised and unsupervised classification or regression problems. While in feature subset selection a subset of the original attributes is extracted, dimensionality reduction in general produces linear combinations of the original attribute set. In this paper we investigate the relationship between several attribute space reduction techniques and the resulting classification accuracy for two very different application areas. On the one hand, we consider e-mail filtering, where the feature space contains various properties of e-mail messages, and on the other hand, we consider drug discovery problems, where quantitative representations of molecular structures are encoded in terms of information-preserving descriptor values. Subsets of the original attributes constructed by filter and wrapper techniques as well as subsets of linear combinations of the original attributes constructed by three different variants of the principle component analysis (PCA) are compared in terms of the classification performance achieved with various machine learning algorithms as well as in terms of runtime performance. We successively reduce the size of the attribute sets and investigate the changes in the classification results. Moreover, we explore the relationship between the variance captured in the linear combinations within PCA and the resulting classification accuracy. The results show that the classification accuracy based on PCA is highly sensitive to the type of data and that the variance captured the principal components is not necessarily a vital indicator for the classification performance.

A Comparison of Classification Accuracy Achieved with Wrappers, Filters and PCA

On the Relationship Between Feature Selection and Classification Accuracy

Supervised Linear Dimension-Reduction Methods: Review, Extensions, and Comparisons

Reducing Error of Tumor Classification by Using Dimension Reduction with Feature Selection

A Hybrid PCA-LDA Model for Dimension Reduction.

PCA-KL: a parametric dimensionality reduction approach for unsupervised metric learning

Improved Algorithms for High-Dimensional Robust Pca

A Comparison Study on Nonlinear Dimension Reduction Methods with Kernel Variations: Visualization, Optimization and Classification.

Representation and Classification of High-Dimensional Biomedical Spectral Data

A General Wrapper Approach to Selection of Class-Dependent Features

A Hybrid Dimension Reduction Based Linear Discriminant Analysis for Classification of High-Dimensional Data

Feature Selection for Classification using Principal Component Analysis and Information Gain

Selecting Subsets of Newly Extracted Features from PCA and PLS in Microarray Data Analysis

A Comparative Study on using Principle Component Analysis with Different Text Classifiers

Supervised Discriminative Sparse PCA with Adaptive Neighbors for Dimensionality Reduction

Feature Selection For Identifying Critical Variables Of Principal Components Based On K-Nearest Neighbor Rule

Combined Supervised Information with PCA Via Discriminative Component Selection

Joint Principal Component and Discriminant Analysis for Dimensionality Reduction

Modified Principal Component Analysis: an Integration of Multiple Similarity Subspace Models.

L-PCA-based dimensionality reduction algorithm for high dimension images

Comparative Analysis of 2D-PCA Based Dimensionality Reduction and Feature Extraction