Abstract:The feature selection in a traditional binary classification algorithm is always used in the stage of dataset preprocessing, which makes the obtained features not necessarily the best ones for the classification algorithm, thus affecting the classification performance. For a traditional rule-based binary classification algorithm, classification rules are usually deterministic, which results in the fuzzy information contained in the rules being ignored. To do so, this paper employs iterative feature selection in fuzzy rule-based binary classification. The proposed algorithm combines feature selection based on fuzzy correlation family with rule mining based on biclustering. It first conducts biclustering on the dataset after feature selection. Then it conducts feature selection again for the biclusters according to the feedback of biclusters evaluation. In this way, an iterative feature selection framework is build. During the iteration process, it stops until the obtained bicluster meets the requirements. In addition, the rule membership function is introduced to extract vectorized fuzzy rules from the bicluster and construct weak classifiers. The weak classifiers with good classification performance are selected by Adaptive Boosting and the strong classifier is constructed by "weighted average". Finally, we perform the proposed algorithm on different datasets and compare it with other peers. Experimental results show that it achieves good classification performance and outperforms its peers.
What problem does this paper attempt to address?
The main problems this paper attempts to address are:
1. **Separation of Feature Selection and Classification Algorithms**: In traditional binary classification algorithms, feature selection is usually performed during the data preprocessing stage. This leads to the selected features not necessarily being the most suitable for the classification algorithm, thereby affecting classification performance. Additionally, the lack of interaction between feature selection and classification algorithms makes the feature selection process often blind, unable to be optimized according to the needs of the classification algorithm.
2. **Limitations of Deterministic Rules**: For rule-based traditional binary classification algorithms, classification rules are usually deterministic, meaning that the combination of features in the rules corresponds to a certain class of samples. However, in actual binary classification problems, the same set of feature values may correspond to different types of samples, i.e., the rules are actually fuzzy. Therefore, extracting deterministic rules may lead to information loss and reduce classification performance.
To address these issues, the paper proposes a binary classification algorithm that combines an iterative feature selection framework and fuzzy rules. Specifically, the algorithm is implemented through the following steps:
- **Iterative Feature Selection Framework**: First, a feature selection method based on fuzzy relevance families is used to select features from the original dataset. Then, a heuristic biclustering algorithm is used to bicluster the selected feature dataset to mine classification rules. By evaluating the support \( S \) of the biclusters, it is determined whether the biclusters meet the requirements. If not, feature selection based on fuzzy relevance families is performed again, and biclustering search is repeated. This process iterates continuously until the optimal biclusters are obtained.
- **Fuzzy Rule Extraction**: Rule membership functions are introduced to extract fuzzy rules from the biclusters. These fuzzy rules are classified according to the membership functions and weak classifiers are constructed.
- **Ensemble Learning**: The AdaBoost algorithm is used to test the weak classifiers, select well-performing weak classifiers, and construct a strong classifier through weighted voting.
Through the above methods, this paper aims to improve the performance of binary classification tasks, especially when dealing with large-scale, high-dimensional datasets, effectively reducing computational complexity and improving classification accuracy.