Abstract:Information theory has been widely used to evaluate the relevance and redundancy of features in feature selection. The traditional feature selection methods based on information theory focus on classification-specific feature selection. Classification-specific feature selection considers all classes together and therefore may not be able to select a desirable feature subset for a particular class. In contrast, class-specific feature selection can choose an appropriate feature subset for each class. The main goal of this paper is to propose a class-specific feature selection method based on information theory. In this regard, we first develop a class-specific feature evaluation criterion called Class-Specific Maximal Dynamic Correlation Change and Minimal Redundancy (CSMDCCMR), which consists of three items. The first item characterizes the correlation between a candidate feature and a particular class. The second item quantifies the dynamic changes in the relevance of a candidate feature and selected features with respect to a particular class. The third item expresses the redundancy between a candidate feature and selected features with respect to a particular class. We then design a class-specific feature selection algorithm by combining the CSMDCCMR criterion with a sequential forward search strategy, which can be employed to select a suitable feature subset for each class. Finally, we carry out extensive experiments to compare our class-specific feature selection method with nine representative classification-specific feature selection methods based on information theory and six state-of-the-art class-specific feature selection methods. The experimental results show that our class-specific feature selection method achieves better classification performance than these comparison methods.

A Fast Algorithm for Feature Selection in Conditional Maximum Entropy Modeling

A Progressive Feature Selection Algorithm for Ultra Large Feature Spaces.

A Fast Feature Selection Algorithm by Accelerating Computation of Fuzzy Rough Set-Based Information Entropy

A Feature Selection Method Based on Feature Grouping and Genetic Algorithm

Feature Selection with Attributes Clustering by Maximal Information Coefficient

Learning to Maximize Mutual Information for Dynamic Feature Selection

Feature Genes Selection Based on Fuzzy Neighborhood Conditional Entropy

A Feature Matching Method: Sparse Feature Tree

Accelerating information entropy-based feature selection using rough set theory with classified nested equivalence classes

A Constrained Feature Selection Approach Based on Feature Clustering and Hypothesis Margin Maximization

A many-objective integrated evolutionary algorithm for feature selection in anomaly detection

Entropy based measure and its algorithms for scalable feature selection

Feature selection for multiset-valued data based on fuzzy conditional information entropy using iterative model and matrix operation

Incremental Subgraph Feature Selection for Graph Classification.

Performance Optimization of a Fuzzy Entropy based Feature Selection and Classification Framework

Incremental feature selection approach to multi-dimensional variation based on matrix dominance conditional entropy for ordered data set

A two-way accelerator for feature selection using a monotonic fuzzy conditional entropy

Class-specific feature selection via maximal dynamic correlation change and minimal redundancy

Combined maximum entropy language model using different feature sets

An information-theoretic feature selection method based on estimation of Markov blanket

Feature Selection Approach Based on Improved Fuzzy C-Means with Principle of Refined Justifiable Granularity