Abstract:Feature selection effectively reduces the dimensionality of data. For feature selection, rough set theory offers a systematic theoretical framework based on consistency measures, of which information entropy is one of the most important significance measures of attributes. However, an information-entropy-based significance measure is computationally expensive and requires repeated calculations. Although many accelerating strategies have been proposed thus far, there remains a bottleneck when using an information-entropy-based feature selection algorithm to handle large-scale datasets with high dimensions. In this study, we introduce a classified nested equivalence class (CNEC)-based approach to calculate the information-entropy-based significance for feature selection using rough set theory. The proposed method extracts knowledge of the reducts of a decision table to reduce the universe and construct CNECs. By exploring the properties of different types of CNECs, we can not only accelerate both outer and inner significance calculation by discarding useless CNECs but also effectively decrease the number of inner significance calculations by using one type of CNECs. The use of CNECs is shown to significantly enhance three representative entropy-based feature selection algorithms that use rough set theory. The feature subset selected by the CNEC-based algorithms is the same as that selected by algorithms using the original definition of information entropies. Experiments conducted using 31 datasets from multiple sources, such as the UCI repository and KDD Cup competition, including large-scale and high-dimensional datasets, confirm the efficiency and effectiveness of the proposed method.

Discretization Algorithm Based on Effective Information Ratio

Novel algorithm for attribute reduction based on mutual-information gain ratio

Immune Algorithm for Discretization of Decision Systems in Rough Set Theory

Reduct Algorithm Based on Information Entropy and Rough Set Theory

A discretization algorithm based on information distance criterion and ant colony optimization algorithm for knowledge extracting on industrial database

A New Discretization Approach of Continuous Attributes

A novel discretization algorithm based on multi-scale and information entropy

Dynamic Discretization: A Combination Approach

An ICA-Based multivariate discretization algorithm

Discrete K-L Transform And Information Feature Analysis

Method of Discretization of Continuous Attributes of Decision Table

A rough set based clustering algorithm and the information theoretical approach to refine clusters

Attribute Reduction Based on Improved Information Entropy

On the Effectiveness of Discretizing Quantitative Attributes in Linear Classifiers

Discretization of Continuous Attributes in Rough Set Theory Based on Information Entropy

A divide-and-conquer discretization algorithm

Discretizing Numerical Attributes: An Analysis of Human Perceptions

ITCI: an Information Theory Based Classification Algorithm for Incomplete Data.

Accelerating information entropy-based feature selection using rough set theory with classified nested equivalence classes

A novel attribute reduction approach using coverage-credibility-based rough decision entropy for interval-valued data

A New Supervised Discretization Method Based on Bayes Measure