Consistent Matrix: A Feature Selection Framework for Large-Scale Data Sets
Tian Yang,Yuan-Jiang Li,Yuhua Qian,Fei-Yue Wang
DOI: https://doi.org/10.1109/tfuzz.2023.3275635
IF: 12.253
2023-01-01
IEEE Transactions on Fuzzy Systems
Abstract:Large-scale data processing based on limited computing resources has always been a difficult problem in data mining, where feature selection is often used as an effective data compressing mechanism. For granular computing of big data, discernibility matrix and dependency degree are the most representative methods for matrix-based and feature importance degree-based feature selection, respectively. However, their temporal and space complexities are high and often lead to poor performance. In this paper, a novel feature selection framework for large scale data processing with linear complexities was proposed for the first time. Firstly, a much more concise fuzzy granule set, called fuzzy arithmetic covering, was introduced to reduce computational costs. Then a new matrix-based feature selection framework, namely consistent matrix, was proposed for general rough set models. As a result, a heuristic attribute reduction algorithm, i.e., HARCM, was designed accordingly. Compared with six state-of-the-art algorithms for feature selection, the average running time of the newly proposed algorithm was reduced up to 2913 times, with a comparable or even better classification performance.
computer science, artificial intelligence,engineering, electrical & electronic