Abstract:While the classical KNN (k nearest neighbor) shares its avoidance of the consistent distribution assumption between training and testing samples to achieve fast prediction, it still faces two challenges: (a) its generalization ability heavily depends on an appropriate number k of nearest neighbors; (b) its prediction behavior lacks interpretability. In order to address the two challenges, a novel Bayes-decisive linear KNN with adaptive nearest neighbors (i.e., BLA-KNN) is proposed to obtain the following three merits: (a) a diagonal matrix is introduced to adaptively select the nearest neighbors and simultaneously improve the generalization capability of the proposed BLA-KNN method; (b) the proposed BLA-KNN method owns the group effect, which inherits and extends the group property of the sum of squares for total deviations by reflecting the training sample class-aware information in the group effect regularization term; (c) the prediction behavior of the proposed BLA-KNN method can be interpreted from the Bayes-decision-rule perspective. In order to do so, we first use a diagonal matrix to weigh each training sample so as to obtain the importance of the sample, while constraining the importance weights to ensure that the adaptive k value is carried out efficiently. Second, we introduce a class-aware information regularization term in the objective function to obtain the nearest neighbor group effect of the samples. Finally, we introduce linear expression weights related to the distance measure between the testing and training samples in the regularization term to ensure that the interpretation of Bayes-decision-rule can be performed smoothly. We also optimize the proposed objective function using an alternating optimization strategy. We experimentally demonstrate the effectiveness of the proposed BLA-KNN method by comparing it with 7 comparative methods on 15 benchmark datasets.

Adaptive Three-way KNN Classifier Using Density-based Granular Balls

GB-DBSCAN: A fast granular-ball based DBSCAN clustering algorithm

GBG++: A Fast and Stable Granular Ball Generation Method for Classification

An Efficient and Adaptive Granular-ball Generation Method in Classification Problem

GBCT: An Efficient and Adaptive Granular-Ball Clustering Algorithm for Complex Data

GNN-DBSCAN: A new density-based algorithm using grid and the nearest neighbor

RW.KNN: a proposed random walk KNN algorithm for multi-label classification.

A robust three-way classifier with shadowed granular-balls based on justifiable granularity

A framework of granular-ball generation for classification via granularity tuning

An adaptive density clustering approach with multi-granularity fusion

A Novel Density Peaks Clustering Algorithm Based on K Nearest Neighbors with Adaptive Merging Strategy

Generation of Granular-Balls for Clustering Based on the Principle of Justifiable Granularity

GBMST: An Efficient Minimum Spanning Tree Clustering Based on Granular-Ball Computing

Bayes-Decisive Linear KNN with Adaptive Nearest Neighbors

Density Peaks Clustering by Granular Computing with Label Propagation

Granular Ball K-Class Twin Support Vector Classifier

K-Ns: A Classifier by the Distance to the Nearest Subspace.

VPGB: A granular-ball based model for attribute reduction and classification with label noise

Data Classification Based on Density Function Estimated by Radial Basis Function

Training Radial Basis Function Classifiers with Gaussian Kernel Clustering and Fuzzy Decision Technique

A novel density-based clustering algorithm using nearest neighbor graph