Abstract:The ultimate goal of supervised feature selection is to select a feature subset such that the prediction accuracy is maximized. Most of the existing methods, such as filter and embedded models, can be viewed as using approximate objectives which are different from the prediction accuracy. The wrapper models maximize the prediction accuracy directly, but the optimization has very high computational complexity. To address the limitations, we present an ordinal optimization perspective for feature selection (OOFS). Feature subset evaluation is formulated as a system simulation process with randomness. Supervised feature selection becomes maximizing the expected performance of the system, where ordinal optimization can be applied to identify a set of order-good-enough solutions with much reduced complexity and parallel computing. These solutions correspond to the really good enough (value-good-enough) solutions when the solution space structure, characterized by ordered performance curve (OPC), exhibits concave shapes. We analyze that this happens in some important applications such as image classification, where a large number features have relatively similar abilities in discrimination. We further improve the OOFS method with a feature scoring algorithm, called OOFSs. We prove that, when the performance difference of solutions increases monotonically with respect to the solution difference, the expectation of the scores reflects useful information for estimating the globally optimal solution. Experimental results in sixteen real-world datasets show that our method provides a good trade-off between prediction accuracy and computational complexity.

Feature Selection for Monotonic Classification

Feature Selection for Monotonic Classification Via Maximizing Monotonic Dependency

Large-margin Feature Selection for Monotonic Classification

An Improved Feature Selection Algorithm for Ordinal Classification.

Rank Entropy-Based Decision Trees for Monotonic Classification

Unsupervised Feature Selection with Ordinal Locality.

Feature Selection with Conditional Mutual Information Considering Feature Interaction

Feature Selection Method on Imbalanced Text

A New Supervised Feature Selection Method for Pattern Classification.

Budget constrained non-monotonic feature selection

Monotonic classification: an overview on algorithms, performance measures and data sets

Fraudulent Firm Classification Using Monotonic Classification Techniques

Supervised Feature Subset Selection with Ordinal Optimization

Fuzzy Mutual Information Based Min-Redundancy and Max-Relevance Heterogeneous Feature Selection

Weak Monotonicity With Trend Analysis for Unsupervised Feature Evaluation

Information-theoretic Partially Labeled Heterogeneous Feature Selection Based on Neighborhood Rough Sets

Conditional mutual information-based feature selection analyzing for synergy and redundancy

Neurodynamics-driven supervised feature selection.

Metric learning for monotonic classification: turning the space up to the limits of monotonicity

Information Entropy for Ordinal Classification

Fuzzy Monotonic K-Nearest Neighbor Versus Monotonic Fuzzy K-Nearest Neighbor