Abstract:Feature selection approaches based on mutual information can be roughly categorized into two groups. The first group minimizes the redundancy of features between each other. The second group maximizes the new classification information of features providing for the selected subset. A critical issue is that large new information does not signify little redundancy, and vice versa. Features with large new information but with high redundancy may be selected by the second group, and features with low redundancy but with little relevance with classes may be highly scored by the first group. Existing approaches fail to balance the importance of both terms. As such, a new information term denoted as Independent Classification Information is proposed in this paper. It assembles the newly provided information and the preserved information negatively correlated with the redundant information. Redundancy and new information are properly unified and equally treated in the new term. This strategy helps find the predictive features providing large new information and little redundancy. Moreover, independent classification information is proved as a loose upper bound of the total classification information of feature subset. Its maximization is conducive to achieve a high global discriminative performance. Comprehensive experiments demonstrate the effectiveness of the new approach.

Mutual Information Criteria for Feature Selection

Feature Selection with Conditional Mutual Information Considering Feature Interaction

Feature Selection Using Mutual Information: An Experimental Study

Feature Selection With Dynamic Mutual Information

Input Feature Selection Method Based on Feature Set Equivalence and Mutual Information Gain Maximization

Feature selection based on Mutual Information in supervised learning

Feature Selection by Maximizing Independent Classification Information.

Feature Selection by Maximizing Part Mutual Information

A Hypergraph-Based Approach to Feature Selection

Feature selection based on mutual information with correlation coefficient

Feature Selection Based on Data Clustering

Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy

Conditional mutual information-based feature selection analyzing for synergy and redundancy

An Interaction-Enhanced Feature Selection Algorithm

Higher Order Mutual Information Approximation for Feature Selection

Latest CHORUS and NOMAD results

A Filter Approach to Feature Selection Based on Mutual Information

A Feature Selection Algorithm Based on Equal Interval Division and Conditional Mutual Information

Class-specific Mutual Information Variation for Feature Selection.

Feature selection based on complementarity of feature classification capability

Feature Selection for Monotonic Classification