Abstract:Neural networks in many varieties are touted as very powerful machine learning tools because of their ability to distill large amounts of information from different forms of data, extracting complex features and enabling powerful classification abilities. In this study, we use neural networks to extract features from both images and numeric data and use these extracted features as inputs for other machine learning models, namely support vector machines (SVMs) and k-nearest neighbor classifiers (KNNs), in order to see if neural-network-extracted features enhance the capabilities of these models. We tested 7 different neural network architectures in this manner, 4 for images and 3 for numeric data, training each for varying lengths of time and then comparing the results of the neural network independently to those of an SVM and KNN on the data, and finally comparing these results to models of SVM and KNN trained using features extracted via the neural network architecture. This process was repeated on 3 different image datasets and 2 different numeric datasets. The results show that, in many cases, the features extracted using the neural network significantly improve the capabilities of SVMs and KNNs compared to running these algorithms on the raw features, and in some cases also surpass the performance of the neural network alone. This in turn suggests that it may be a reasonable practice to use neural networks as a means to extract features for classification by other machine learning models for some datasets.

A Comparative Study of Two Kernel Ideas for Nonlinear Feature Extraction

On Feature Extraction Via Kernels

Modified Kernel-Based Nonlinear Feature Extraction [face Recognition Example]

Bilinear Analysis for Kernel Selection and Nonlinear Feature Extraction

MODIFIED KERNEL-BASED NONLINEAR FEATURE EXTRACTION

Developments and Applications of Nonlinear Principal Component Analysis – a Review

Nonlinear Component Analysis as a Kernel Eigenvalue Problem

Adaptive kernel subspace method for speeding up feature extraction

Nonlinear Pocket Algorithm with Kernels

An Optimal Kernel Feature Extractor and Its Application to EEG Signal Classification

Kernel Generalized Nonlinear Discriminant Analysis Algorithm For Pattern Recognition

A New Nonlinear Feature Extraction Method for Face Recognition.

Data-Dependent Kernel Discriminant Analysis for Feature Extraction and Classification

Image Quality Assessment Method Based on Nonlinear Feature Extraction in Kernel Space

A Review of Kernel Methods for Feature Extraction in Nonlinear Process Monitoring

Nonlinear Process Monitoring Using Improved Kernel Principal Component Analysis

Face recognition using a kernel fractional-step discriminant analysis algorithm

<Bold>An Implementation Framework For Kernel Methods With High-Dimensional Patterns</Bold>

Phase Space Reconstruction of Nonlinear Time Series Based on Kernel Method

Adaptive quasiconformal kernel discriminant analysis

Examining the Use of Neural Networks for Feature Extraction: A Comparative Analysis using Deep Learning, Support Vector Machines, and K-Nearest Neighbor Classifiers