Abstract:While the classical KNN (k nearest neighbor) shares its avoidance of the consistent distribution assumption between training and testing samples to achieve fast prediction, it still faces two challenges: (a) its generalization ability heavily depends on an appropriate number k of nearest neighbors; (b) its prediction behavior lacks interpretability. In order to address the two challenges, a novel Bayes-decisive linear KNN with adaptive nearest neighbors (i.e., BLA-KNN) is proposed to obtain the following three merits: (a) a diagonal matrix is introduced to adaptively select the nearest neighbors and simultaneously improve the generalization capability of the proposed BLA-KNN method; (b) the proposed BLA-KNN method owns the group effect, which inherits and extends the group property of the sum of squares for total deviations by reflecting the training sample class-aware information in the group effect regularization term; (c) the prediction behavior of the proposed BLA-KNN method can be interpreted from the Bayes-decision-rule perspective. In order to do so, we first use a diagonal matrix to weigh each training sample so as to obtain the importance of the sample, while constraining the importance weights to ensure that the adaptive k value is carried out efficiently. Second, we introduce a class-aware information regularization term in the objective function to obtain the nearest neighbor group effect of the samples. Finally, we introduce linear expression weights related to the distance measure between the testing and training samples in the regularization term to ensure that the interpretation of Bayes-decision-rule can be performed smoothly. We also optimize the proposed objective function using an alternating optimization strategy. We experimentally demonstrate the effectiveness of the proposed BLA-KNN method by comparing it with 7 comparative methods on 15 benchmark datasets.

SNNB: A Selective Neighborhood Based Naïve Bayes for Lazy Learning.

A Study of Selective Neighborhood-Based Nai"ve Bayes for Efficient Lazy Learning.

Enhancing SNNB with Local Accuracy Estimation and Ensemble Techniques

Evolutionary Lazy Learning for Naive Bayes Classification

Sparse Bayesian Approach to Fast Learning Network for Multiclassification.

Locally Weighted Learning: How and when Does It Work in Bayesian Networks?

Efficient heuristics for learning scalable Bayesian network classifier from labeled and unlabeled data

A Bayesian Network nearest k-labels method for Multi-label classification

Locally Weighted Learning for Naive Bayes Classifier

Network-based Naive Bayes Model for Social Network

Learning semi-lazy Bayesian network classifier under the c.i.i.d assumption

A Classifier Selection Strategy for Lazy Bayesian Rules Based on Local Accuracy Estimation

Local Naive Bayes Nearest Neighbor for Image Classification

Lazy Bayesian Rules: A Lazy Semi-Naive Bayesian Learning Technique Competitive to Boosting Decision Trees

Bayes-Decisive Linear KNN with Adaptive Nearest Neighbors

Efficient Heuristics for Learning Bayesian Network from Labeled and Unlabeled Data

Improved Weighted Naive Bayes Classification Algorithm Based on Attribute Selection

Lazy Learning Based Double Layer Nave Bayesian Classifier

Bi-Selection of Instances and Features Based on Neighborhood Importance Degree

A Naïve Bayes Regularized Logistic Regression Estimator for Low-dimensional Classification

Image Classification Based on Low-Rank Matrix Recovery and Naive Bayes Collaborative Representation

SNNB: A Selective Neighborhood Based Na&iuml;ve Bayes for Lazy Learning.