Abstract:The ultimate goal of supervised feature selection is to select a feature subset such that the prediction accuracy is maximized. Most of the existing methods, such as filter and embedded models, can be viewed as using approximate objectives which are different from the prediction accuracy. The wrapper models maximize the prediction accuracy directly, but the optimization has very high computational complexity. To address the limitations, we present an ordinal optimization perspective for feature selection (OOFS). Feature subset evaluation is formulated as a system simulation process with randomness. Supervised feature selection becomes maximizing the expected performance of the system, where ordinal optimization can be applied to identify a set of order-good-enough solutions with much reduced complexity and parallel computing. These solutions correspond to the really good enough (value-good-enough) solutions when the solution space structure, characterized by ordered performance curve (OPC), exhibits concave shapes. We analyze that this happens in some important applications such as image classification, where a large number features have relatively similar abilities in discrimination. We further improve the OOFS method with a feature scoring algorithm, called OOFSs. We prove that, when the performance difference of solutions increases monotonically with respect to the solution difference, the expectation of the scores reflects useful information for estimating the globally optimal solution. Experimental results in sixteen real-world datasets show that our method provides a good trade-off between prediction accuracy and computational complexity.

Feature Selection for Monotonic Classification Via Maximizing Monotonic Dependency

Feature Selection for Monotonic Classification

Large-margin Feature Selection for Monotonic Classification

An Improved Feature Selection Algorithm for Ordinal Classification.

Feature Selection Based on Dependency Margin

Feature Selection with Conditional Mutual Information Considering Feature Interaction

Budget constrained non-monotonic feature selection

Neurodynamics-driven supervised feature selection.

Fraudulent Firm Classification Using Monotonic Classification Techniques

An Adaptive Feature Selection Method for Multi-Class Classification.

A Variance Minimization Criterion to Feature Selection Using Laplacian Regularization

Monotonic classification: an overview on algorithms, performance measures and data sets

A Constrained Feature Selection Approach Based on Feature Clustering and Hypothesis Margin Maximization

Supervised Feature Selection via Collaborative Neurodynamic Optimization

Weak Monotonicity With Trend Analysis for Unsupervised Feature Evaluation

An Improved Feature Selection Algorithm for Fault Level Identification

Effective Learning with Joint Discriminative and Representative Feature Selection

Metric learning for monotonic classification: turning the space up to the limits of monotonicity

A neurodynamic optimization approach to supervised feature selection via fractional programming

Supervised Feature Subset Selection with Ordinal Optimization

Feature Selection Via Scaling Factor Integrated Multi-Class Support Vector Machines