Abstract:Feature selection can greatly enhance the performance of a learning algorithm when dealing with a high dimensional data set. The filter method and the wrapper method are the two most commonly approaches. However, these approaches have limitations. The filter method uses independent evaluation to evaluate and select features, which is computationally efficient but less accurate than the wrapper method. The wrapper method uses a predetermined classifier to compute the evaluation, which can afford high accuracy for particular classifiers, but is computationally expensive. In this study, we introduce a new feature selection method that we refer to as the large margin hybrid algorithm for feature selection (LMFS). In this method, we first utilize a new distance-based evaluation function, in which ideally samples from the same class are close together, whereas samples from other classes are far apart, and a weighted bootstrapping search strategy to find a set of candidate feature subsets. Then, we use a specific classifier and cross-validation to select the final feature subset from the candidate feature subsets. Six vibrational spectroscopic data sets and three different classifiers, namely k-nearest neighbors, partial least squares discriminant analysis and least squares support vector machine were used to validate the performance of the LMFS method. The results revealed that LMFS can effectively overcome the over-fitting between the optimal feature subset and a given classifier. Compared with the filter and wrapper methods, the features selected by the LMFS method have better classification performance and model interpretation. Furthermore, LMFS can effectively overcomes the impact of classifier complexity on computational time, and distance-based classifiers were found to be more suitable for selecting the final subset in LMFS.

A Feature Selection Method Based on ReliefF Evaluation and Complementary Coefficient

A feature selection algorithm based on classification complementarity

An Unsupervised Feature Selection Method Based on Improved ReliefF and Bisecting K-means

A New Feature Selection Algorithm Based on Relevance, Redundancy and Complementarity.

An Improved Combination Feature Selection Based on ReliefF and Genetic Algorithm

New hybrid feature selection method RRK

An adaption of relief for redundant feature elimination

A Hybrid Feature Selection Method Based on Binary State Transition Algorithm and ReliefF

An Improved Wrapper Method for Feature Selection

A feature selection approach based on NSGA-II with ReliefF

Relief-Based Feature Selection: Introduction and Review

A New Feature Selection Method Based on a Validity Index of Feature Subset

A hybrid feature selection method using both filter and wrapper in mammography CAD

A Novel Hybrid Algorithm for Feature Selection

A Hybrid Feature Selection Approach by Correlation-Based Filters and SVM-RFE

A New Hybrid Filter/wrapper Algorithm for Feature Selection in Classification.

A new filter feature selection algorithm for classification task by ensembling pearson correlation coefficient and mutual information

A Two-Phase Feature Selection Method Using Both Filter and Wrapper

A Hybrid Feature Selection Approach Based on ReliefF-FC-SS Algorithm for Multi-feature Data

Two-step Based Feature Selection Method for Filtering Redundant Information

Feature Selection using ReliefF Algorithm