Abstract:Feature selection (FS) is an indispensable activity in machine learning , whose purpose is to identify relevant predictive values from a high-dimensional feature space to improve performance and reduce model learning time. However, the large increase in feature dimensions poses a great challenge to FS methods. Therefore, a multi-objective optimization algorithm consisting of an Equilibrium Optimizer (EO) and NSGA-III was proposed to solve the FS problem with high-dimensional data. Through S-shaped, V-shaped and U-shaped transfer functions, the conversion from real number coding to binary coding is realized to solve discrete problems, and the influence of these three transfer functions on the effect of FS is compared. In addition, the algorithm optimizes the population in the binary search space by building an external archive , and realizes the selection and optimization of external archive individuals based on the clustering strategy. The KNN classifier was used to realize the classification progress. The simulation experiments are divided into two groups with 18 medium and high dimensional data. The first group analyzes the optimization effect of the proposed framework under three transfer functions. The second group of experiments selects the algorithm that wins in the first group of experiments and compares it with eleven classical multi-objective optimization algorithms. The evaluation criteria includes two optimization objectives of the FS problem and the optimization indices of HV and IGD. The first set of experiments showed that the U-shaped transfer function family performed best in the FS problem, with U3 being the most excellent, followed by V-shaped and S-shaped. Compared to other multi-objective optimization algorithms, the simulation results also confirm the effectiveness of the proposed strategy.

Gradient Boosted Feature Selection.

Feature Selection and Parameter Optimization for Support Vector Machines: A New Approach Based on Genetic Algorithm with Feature Chromosomes.

A Feature Selection Method Based on Feature Grouping and Genetic Algorithm

GBRUN: A Gradient Search-based Binary Runge Kutta Optimizer for Feature Selection

BGOA-TVG: Binary Grasshopper Optimization Algorithm with Time-Varying Gaussian Transfer Functions for Feature Selection

Feature selection based on artificial bee colony and gradient boosting decision tree

Evaluation of Feature Selection Methods Using Bagging and Boosting Ensemble Techniques on High Throughput Biological Data

Gradient-based Laplacian Feature Selection

Boosting: An Ensemble Learning Tool for Compound Classification and QSAR Modeling

Invariant optimal feature selection: A distance discriminant and feature ranking based solution

GAdaBoost: Accelerating Adaboost Feature Selection with Genetic Algorithms

A Modified Sequential Deep Floating Search Algorithm For Feature Selection

Feature Gradients: Scalable Feature Selection via Discrete Relaxation

Explorative Binary Gray Wolf Optimizer with Quadratic Interpolation for Feature Selection

DimBoost

Multi-objective optimization algorithm based on clustering guided binary equilibrium optimizer and NSGA-III to solve high-dimensional feature selection problem

Unbiased Gradient Boosting Decision Tree with Unbiased Feature Importance

Bhattacharyya boosting

Clustering-Guided Sparse Structural Learning for Unsupervised Feature Selection

A problem-specific non-dominated sorting genetic algorithm for supervised feature selection

A Contrast Based Feature Selection Algorithm for High-dimensional Data set in Machine Learning