Abstract:Feature selection aims to eliminate redundant or irrelevant variables from input data to reduce computational cost, provide a better understanding of data and improve prediction accuracy. Majority of the existing filter methods utilize a single feature-ranking technique, which may overlook some important assumptions about the underlying regression function linking input variables with the output. In this paper, we propose a novel feature selection framework that combines clustering of variables with multiple feature-ranking techniques for selecting an optimal feature subset. Different feature-ranking methods typically result in selecting different subsets, as each method has its own assumption about the regression function linking input variables with the output. Therefore, we employ multiple feature-ranking methods having disjoint assumption about the regression function. The proposed approach has a feature ranking module to identify relevant features and a clustering module to eliminate redundant features. First, input variables are ranked using regression coefficients obtained by training $L1$ regularized Logistic Regression, Support Vector Machine and Random Forests models. Those features which are ranked lower than a certain threshold are filtered-out. The remaining features are grouped into clusters using an exemplar-based clustering algorithm, which identifies data-points that exemplify the data better, and associates each data-point with an exemplar. We use both linear correlation coefficients and information gain for measuring the association between a data-point and its corresponding exemplar. From each cluster the highest ranked feature is selected as a delegate, and all delegates from the three ranked lists are combined into the final feature set using union operation. Empirical results over a number of real-world data sets confirm the hypothesis that combining features selected using multiple heterogeneous methods results in a more robust feature set and improves prediction accuracy. As compared to other feature selection approaches evaluated, features selected using linear correlation-based multi-filter feature selection achieved the best classification accuracy with 98.7%, 100%, 92.3% and 100% for Ionosphere, Wisconsin Breast Cancer, Sonar and Wine data sets respectively.

RANDOM FOREST AND SUPPORT VECTOR MACHINE ON FEATURES SELECTION FOR REGRESSION ANALYSIS

Predicting Disease Risks Using Feature Selection Based on Random Forest and Support Vector Machine.

Application of Random Forest Algorithm on Feature Subset Selection and Classification and Regression

Support Vector Machine-Recursive Feature Elimination for Localized Feature Selection

RANDOM FORESTS:AN IMPORTANT FEATURE GENES SELECTION METHOD OF TUMOR

Accelerated Recursive Feature Elimination Based on Support Vector Machine for Key Variable Identification

A comparison of random forest variable selection methods for classification prediction modeling

Speaker-independent Speech Emotion Recognition Based on Random Forest Feature Selection Algorithm

A Hybrid Feature Selection Approach by Correlation-Based Filters and SVM-RFE

Selection Features and Support Vector Machine for Credit Card Risk Identification

A Novel Random Forests-Based Feature Selection Method for Microarray Expression Data Analysis

An Optimized Support Vector Machine Intelligent Technique Using Optimized Feature Selection Methods: Evidence from Chinese Credit Approval Data

A Multi-class Feature Selection Algorithm Based on Support Vector Machine

Feature subset selection for support vector machines through sensitivity analysis

Evaluation of Feature Selection Methods for Object-Based Land Cover Mapping of Unmanned Aerial Vehicle Imagery Using Random Forest and Support Vector Machine Classifiers

CURE-SMOTE algorithm and hybrid algorithm for feature selection and parameter optimization based on random forests

Assessment of feature selection for student academic performance through machine learning classification

Combining Multiple Feature-Ranking Techniques and Clustering of Variables for Feature Selection

A New Noisy Random Forest Based Method for Feature Selection

Decision Variants for the Automatic Determination of Optimal Feature Subset in RF-RFE