Abstract:Feature selection effectively reduces the dimensionality of data. For feature selection, rough set theory offers a systematic theoretical framework based on consistency measures, of which information entropy is one of the most important significance measures of attributes. However, an information-entropy-based significance measure is computationally expensive and requires repeated calculations. Although many accelerating strategies have been proposed thus far, there remains a bottleneck when using an information-entropy-based feature selection algorithm to handle large-scale datasets with high dimensions. In this study, we introduce a classified nested equivalence class (CNEC)-based approach to calculate the information-entropy-based significance for feature selection using rough set theory. The proposed method extracts knowledge of the reducts of a decision table to reduce the universe and construct CNECs. By exploring the properties of different types of CNECs, we can not only accelerate both outer and inner significance calculation by discarding useless CNECs but also effectively decrease the number of inner significance calculations by using one type of CNECs. The use of CNECs is shown to significantly enhance three representative entropy-based feature selection algorithms that use rough set theory. The feature subset selected by the CNEC-based algorithms is the same as that selected by algorithms using the original definition of information entropies. Experiments conducted using 31 datasets from multiple sources, such as the UCI repository and KDD Cup competition, including large-scale and high-dimensional datasets, confirm the efficiency and effectiveness of the proposed method.

An Incremental Feature Selection Approach Based on Information Entropy for Incomplete Data

An Emerging Fuzzy Feature Selection Method Using Composite Entropy-Based Uncertainty Measure and Data Distribution

Feature Selection with Partition Differentiation Entropy for Large-Scale Data Sets.

Feature Selection with Conditional Mutual Information Considering Feature Interaction

Uncertainty Measure-Based Incremental Feature Selection For Hierarchical Classification

Incremental feature selection approach to multi-dimensional variation based on matrix dominance conditional entropy for ordered data set

Accelerating information entropy-based feature selection using rough set theory with classified nested equivalence classes

Selecting features by utilizing intuitionistic fuzzy Entropy method

Incremental neighborhood entropy-based feature selection for mixed-type data under the variation of feature set

Incremental feature selection based on fuzzy rough sets

Feature selection in mixed data: A method using a novel fuzzy rough set-based information entropy

A Fast Feature Selection Algorithm by Accelerating Computation of Fuzzy Rough Set-Based Information Entropy

Incremental feature selection for dynamic incomplete data using sub-tolerance relations

Fuzzy Entropy Based Max-Relevancy and Min-Redundancy Feature Selection.

A Novel Approach of Rough Conditional Entropy-Based Attribute Selection for Incomplete Decision System

Discernible Neighborhood Counting Based Incremental Feature Selection for Heterogeneous Data

Fuzzy Mutual Information Based Min-Redundancy and Max-Relevance Heterogeneous Feature Selection

Fuzzy Rough Sets-Based Incremental Feature Selection for Hierarchical Classification

Information-theoretic Partially Labeled Heterogeneous Feature Selection Based on Neighborhood Rough Sets

An information-theoretic feature selection method based on estimation of Markov blanket