Abstract:k nearest neighbor (kNN) method is a popular classification method in data mining and statistics because of its simple implementation and significant classification performance. However, it is impractical for traditional kNN methods to assign a fixed k value (even though set by experts) to all test samples. Previous solutions assign different k values to different test samples by the cross validation method but are usually time-consuming. This paper proposes a kTree method to learn different optimal k values for different test/new samples, by involving a training stage in the kNN classification. Specifically, in the training stage, kTree method first learns optimal k values for all training samples by a new sparse reconstruction model, and then constructs a decision tree (namely, kTree) using training samples and the learned optimal k values. In the test stage, the kTree fast outputs the optimal k value for each test sample, and then, the kNN classification can be conducted using the learned optimal k value and all training samples. As a result, the proposed kTree method has a similar running cost but higher classification accuracy, compared with traditional kNN methods, which assign a fixed k value to all test samples. Moreover, the proposed kTree method needs less running cost but achieves similar classification accuracy, compared with the newly kNN methods, which assign different k values to different test samples. This paper further proposes an improvement version of kTree method (namely, k*Tree method) to speed its test stage by extra storing the information of the training samples in the leaf nodes of kTree, such as the training samples located in the leaf nodes, their kNNs, and the nearest neighbor of these kNNs. We call the resulting decision tree as k*Tree, which enables to conduct kNN classification using a subset of the training samples in the leaf nodes rather than all training samples used in the newly kNN methods. This actually reduces running cost of test stage. Finally, the experimental results on 20 real data sets showed that our proposed methods (i.e., kTree and k*Tree) are much more efficient than the compared methods in terms of classification tasks.

Combining KNN algorithm and other classifiers.

RW.KNN: a proposed random walk KNN algorithm for multi-label classification.

Combination of Multiple Nearest Neighbor Classifiers Based on Feature Subset Clustering Method

K-Ns: A Classifier by the Distance to the Nearest Subspace.

Combination of Multiple K-NNCS by Fuzzy Integral

Coarse to Fine K Nearest Neighbor Classifier

Hybrid k-Nearest Neighbor Classifier.

SVM-KNN Classifier--A New Method of Improving the Accuracy of SVM Classifier

Active and Passive Nearest Neighbor Algorithm: A Newly-Developed Supervised Classifier

A Fast Document Classification Algorithm Based on Improved KNN

A New Locally Adaptive K-Nearest Neighbor Algorithm Based on Discrimination Class

2-Stage Instance Selection Algorithm for KNN Based on Nearest Unlike Neighbors

Efficient Knn Classification with Different Numbers of Nearest Neighbors.

A new two-layer nearest neighbor selection method for kNN classifier

Doubly Approximate Nearest Neighbor Classification

A New K-Harmonic Nearest Neighbor Classifier Based on the Multi-Local Means

A Local Mean Representation-based K-Nearest Neighbor Classifier

Evidential K-NN Classification with Enhanced Performance Via Optimizing a Class of Parametric Conjunctive T-Rules

Neighbor selection for multilabel classification

Combination Of Multiple Real-Valued Nearest Neighbor Classifiers Based On Different Feature Subsets With Fuzzy Integral

A Novel Classification Algorithm Based on the Synergy Between Dynamic Clustering with Adaptive Distances and K-Nearest Neighbors