Abstract:Information theory has been widely used to evaluate the relevance and redundancy of features in feature selection. The traditional feature selection methods based on information theory focus on classification-specific feature selection. Classification-specific feature selection considers all classes together and therefore may not be able to select a desirable feature subset for a particular class. In contrast, class-specific feature selection can choose an appropriate feature subset for each class. The main goal of this paper is to propose a class-specific feature selection method based on information theory. In this regard, we first develop a class-specific feature evaluation criterion called Class-Specific Maximal Dynamic Correlation Change and Minimal Redundancy (CSMDCCMR), which consists of three items. The first item characterizes the correlation between a candidate feature and a particular class. The second item quantifies the dynamic changes in the relevance of a candidate feature and selected features with respect to a particular class. The third item expresses the redundancy between a candidate feature and selected features with respect to a particular class. We then design a class-specific feature selection algorithm by combining the CSMDCCMR criterion with a sequential forward search strategy, which can be employed to select a suitable feature subset for each class. Finally, we carry out extensive experiments to compare our class-specific feature selection method with nine representative classification-specific feature selection methods based on information theory and six state-of-the-art class-specific feature selection methods. The experimental results show that our class-specific feature selection method achieves better classification performance than these comparison methods.

Minimum Joint Mutual Information Loss-based Optimal Feature Selection Algorithm

Feature Selection with Conditional Mutual Information Considering Feature Interaction

Multi-label Feature Selection Algorithm Based on Joint Mutual Information of Max-Relevance and Min-Redundancy

Conditional Mutual Information-Based Feature Selection Algorithm for Maximal Relevance Minimal Redundancy

Estimating Conditional Mutual Information for Dynamic Feature Selection

Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy

Feature Selection Algorithm Based On Conditional Dynamic Mutual Information

Class-specific feature selection via maximal dynamic correlation change and minimal redundancy

Feature Selection With Dynamic Mutual Information

Feature Selection Using Mutual Information: An Experimental Study

Fuzzy Mutual Information Based Min-Redundancy and Max-Relevance Heterogeneous Feature Selection

Feature Selection by Maximizing Independent Classification Information.

Fuzzy Entropy Based Max-Relevancy and Min-Redundancy Feature Selection.

Conditional mutual information-based feature selection analyzing for synergy and redundancy

Size Adaptive Selection of Most Informative Features

Class-specific Mutual Information Variation for Feature Selection.

Feature Selection by Maximizing Part Mutual Information

Feature selection and threshold method based on fuzzy joint mutual information

Common Mutual Information Selection Algorithm and Its Application on Combination Forecasting

Fuzzy joint mutual information feature selection based on ideal vector

Higher Order Mutual Information Approximation for Feature Selection