Abstract:Neural networks in many varieties are touted as very powerful machine learning tools because of their ability to distill large amounts of information from different forms of data, extracting complex features and enabling powerful classification abilities. In this study, we use neural networks to extract features from both images and numeric data and use these extracted features as inputs for other machine learning models, namely support vector machines (SVMs) and k-nearest neighbor classifiers (KNNs), in order to see if neural-network-extracted features enhance the capabilities of these models. We tested 7 different neural network architectures in this manner, 4 for images and 3 for numeric data, training each for varying lengths of time and then comparing the results of the neural network independently to those of an SVM and KNN on the data, and finally comparing these results to models of SVM and KNN trained using features extracted via the neural network architecture. This process was repeated on 3 different image datasets and 2 different numeric datasets. The results show that, in many cases, the features extracted using the neural network significantly improve the capabilities of SVMs and KNNs compared to running these algorithms on the raw features, and in some cases also surpass the performance of the neural network alone. This in turn suggests that it may be a reasonable practice to use neural networks as a means to extract features for classification by other machine learning models for some datasets.

Approach for Pre-Extracting Support Vectors Based on K-Nn

A novel tree kernel support vector machine classifier for modeling the relationship between bioactivity and molecular descriptors

Recursive Training Algorithm for One-Class Support Vector Machine Based on Active Set Method

An Incremental Updating Method for Support Vector Machines

CompareSVM: supervised, Support Vector Machine (SVM) inference of gene regularity networks

An On-Line Learning Approach with Support Vector Dormain Classifier

Discriminant analysis via support vectors

Fast extraction strategy of support vector machines

K-Ns: A Classifier by the Distance to the Nearest Subspace.

Approximate Approach to Train SVM on Very Large Data Sets

Pair-wise sequential reduced set for optimization of support vector machines

Mini-batch Quasi-Newton Optimization for Large Scale Linear Support Vector Regression

Deep Support Vectors

Modeling method of least squares support vector regression based on vector base learning

Simplified Support Vector Machine Based on Reduced Vector Set Method

Simplified support vector machines via kernel-based clustering

A Geometric Approach To Train Svm On Very Large Data Sets

Examining the Use of Neural Networks for Feature Extraction: A Comparative Analysis using Deep Learning, Support Vector Machines, and K-Nearest Neighbor Classifiers

Fast Pruning Superfluous Support Vectors in SVMs

Kernel Nearest-Neighbor Algorithm

Neighbors' Distribution Property and Sample Reduction for Support Vector Machines