Abstract:Feature selection is predominantly used in machine learning tasks, such as classification, regression, and clustering. It selects a subset of features (relevant attributes of data points) from a larger set that contributes as optimally as possible to the informativeness of the model. There are exponentially many subsets of a given set, and thus, the exhaustive search approach is only practical for problems with at most a few dozen features. In the past, there have been attempts to reduce the search space using dynamic programming. However, models that consider similarity in pairs of features alongside the quality of individual features do not provide the required optimal substructure. As a result, algorithms, which we will call suboptimal dynamic programming algorithms, find a solution that may deviate significantly from the optimal one. In this paper, we propose an iterative dynamic programming algorithm, which invertsthe order of feature processing in each iteration. Such an alternating approach allows for improving the optimization function by using the score from the previous iteration to estimate the contribution of unprocessed features. The iterative process is proven to converge and terminates when the solution does not change in three successive iterations or when the number of iterations reaches the threshold. Results in more than 95% of tests align with those of the exhaustive search approach, being competitive and often superior to the reference greedy approach. Validation was carried out by comparing the scores of output feature subsets and examining the accuracy of different classifiers learned on these features across nine real-world applications, considering different scenarios with various numbers of features and samples. In the context of feature selection, the proposed algorithm can be characterized as a robust filter method that can improve machine learning models regardless of dataset size. However, we expect that the idea of alternating suboptimal optimization will soon be generalized to tasks beyond feature selection.

A neurodynamic optimization approach to supervised feature selection via fractional programming

Two-timescale neurodynamic approaches to supervised feature selection based on alternative problem formulations

Neurodynamics-driven supervised feature selection.

Supervised Feature Selection via Collaborative Neurodynamic Optimization

A Novel Feature Selection Method Based on Extreme Learning Machine and Fractional-Order Darwinian PSO.

Feature Selection Using a Neural Framework With Controlled Redundancy

A Feature Selection Method Based on Feature Grouping and Genetic Algorithm

A New Alternating Suboptimal Dynamic Programming Algorithm with Applications for Feature Selection

Feature Selection for Monotonic Classification Via Maximizing Monotonic Dependency

A Novel Approach for Stable Selection of Informative Redundant Features from High Dimensional fMRI Data

Feature Selection for Monotonic Classification

Learning to Maximize Mutual Information for Dynamic Feature Selection

Feature Selection with Integrated Relevance and Redundancy Optimization

A Constrained Feature Selection Approach Based on Feature Clustering and Hypothesis Margin Maximization

TSFNFS: two-stage-fuzzy-neighborhood feature selection with binary whale optimization algorithm

Differential Evolution Based Feature Selection: A Niching-based Multi-objective Approach

Feature selection considering feature relevance, redundancy and interactivity for neighborhood decision systems

A Feature Selection Method Using Conditional Correlation Dispersion and Redundancy Analysis

Class-specific feature selection via maximal dynamic correlation change and minimal redundancy

A New Method for Redundancy Analysis in Feature Selection