Abstract:Feature selection is predominantly used in machine learning tasks, such as classification, regression, and clustering. It selects a subset of features (relevant attributes of data points) from a larger set that contributes as optimally as possible to the informativeness of the model. There are exponentially many subsets of a given set, and thus, the exhaustive search approach is only practical for problems with at most a few dozen features. In the past, there have been attempts to reduce the search space using dynamic programming. However, models that consider similarity in pairs of features alongside the quality of individual features do not provide the required optimal substructure. As a result, algorithms, which we will call suboptimal dynamic programming algorithms, find a solution that may deviate significantly from the optimal one. In this paper, we propose an iterative dynamic programming algorithm, which invertsthe order of feature processing in each iteration. Such an alternating approach allows for improving the optimization function by using the score from the previous iteration to estimate the contribution of unprocessed features. The iterative process is proven to converge and terminates when the solution does not change in three successive iterations or when the number of iterations reaches the threshold. Results in more than 95% of tests align with those of the exhaustive search approach, being competitive and often superior to the reference greedy approach. Validation was carried out by comparing the scores of output feature subsets and examining the accuracy of different classifiers learned on these features across nine real-world applications, considering different scenarios with various numbers of features and samples. In the context of feature selection, the proposed algorithm can be characterized as a robust filter method that can improve machine learning models regardless of dataset size. However, we expect that the idea of alternating suboptimal optimization will soon be generalized to tasks beyond feature selection.

Feature Selection by Firefly Algorithm with Improved Initialization Strategy

Hybrid firefly particle swarm optimisation algorithm for feature selection problems

A novel firefly algorithm approach for efficient feature selection with COVID-19 dataset

Feature selection for software defect prediction using an improved firefly algorithm

Novel hybrid firefly algorithm: an application to enhance XGBoost tuning for intrusion detection classification

Feature selection based on dataset variance optimization using Hybrid Sine Cosine – Firehawk Algorithm (HSCFHA)

A comprehensive review of firefly algorithms

Feature selection algorithm based on incremental mutual information and cockroach swarm optimization

TSFNFS: two-stage-fuzzy-neighborhood feature selection with binary whale optimization algorithm

An Improved Firefly Algorithm with Distance-Guided Selection Strategy and Its Application.

Firefly Algorithm: Recent Advances and Applications

Improvement and Optimization of Feature Selection Algorithm in Swarm Intelligence Algorithm Based on Complexity

Hierarchical learning multi-objective firefly algorithm for high-dimensional feature selection

Feature selection using bare-bones particle swarm optimization with mutual information

Improved Salp Swarm Algorithm Based on Particle Swarm Optimization for Feature Selection

RSO: A Novel Reinforced Swarm Optimization Algorithm for Feature Selection

A New Alternating Suboptimal Dynamic Programming Algorithm with Applications for Feature Selection

An intelligent feature selection approach based on moth flame optimization for medical diagnosis

Improved Moth-Flame Optimization Based on Opposition-Based Learning for Feature Selection

A novel quantum grasshopper optimization algorithm for feature selection

MFeature: Towards high performance evolutionary tools for feature selection