Abstract:While the classical KNN (k nearest neighbor) shares its avoidance of the consistent distribution assumption between training and testing samples to achieve fast prediction, it still faces two challenges: (a) its generalization ability heavily depends on an appropriate number k of nearest neighbors; (b) its prediction behavior lacks interpretability. In order to address the two challenges, a novel Bayes-decisive linear KNN with adaptive nearest neighbors (i.e., BLA-KNN) is proposed to obtain the following three merits: (a) a diagonal matrix is introduced to adaptively select the nearest neighbors and simultaneously improve the generalization capability of the proposed BLA-KNN method; (b) the proposed BLA-KNN method owns the group effect, which inherits and extends the group property of the sum of squares for total deviations by reflecting the training sample class-aware information in the group effect regularization term; (c) the prediction behavior of the proposed BLA-KNN method can be interpreted from the Bayes-decision-rule perspective. In order to do so, we first use a diagonal matrix to weigh each training sample so as to obtain the importance of the sample, while constraining the importance weights to ensure that the adaptive k value is carried out efficiently. Second, we introduce a class-aware information regularization term in the objective function to obtain the nearest neighbor group effect of the samples. Finally, we introduce linear expression weights related to the distance measure between the testing and training samples in the regularization term to ensure that the interpretation of Bayes-decision-rule can be performed smoothly. We also optimize the proposed objective function using an alternating optimization strategy. We experimentally demonstrate the effectiveness of the proposed BLA-KNN method by comparing it with 7 comparative methods on 15 benchmark datasets.

The K-Nearest Neighbour-Based GMDH Prediction Model and Its Applications

K-Nearest Neighbor Model Based Short-Term Traffic Flow Prediction Method

A revision for gaussian mixture density decomposition algorithm

Modeling a Delayed Coking Process with GRNN and Double-Chain Based DNA Genetic Algorithm

Modified kernel Hebbian algorithm with application to modeling of hydro-dearomatization process

Improved K nearest neighbor nonparametric regression and its application in short-term traffic flow forecasting

Hybrid Metric K-Nearest Neighbor Algorithm and Applications

A New Algorithm for Large-Scale Geographically Weighted Regression with K-Nearest Neighbors

A Generalized Multivariate Grey Prediction Model DGM(1, N, I) with Unknown Time Delays

Multiple graph kernel learning based on GMDH-type neural network

The Improved GMDH-type Neural Network and Its Application to Forecasting Chaotic Time Series

Kernel Generalized Nonlinear Discriminant Analysis Algorithm For Pattern Recognition

Kernel Nearest-Neighbor Algorithm

Hybrid GMDH deep learning networks – analysis, optimization and applications in forecasting at financial sphere

Kernel Mixture Correntropy Conjugate Gradient Algorithm for Time Series Prediction

Kernelized k-Local Hyperplane Distance Nearest-Neighbor Model for Predicting Cerebrovascular Disease in Patients With End-Stage Renal Disease

KGDA: A Knowledge Graph-based Decomposition Approach for Cellular Traffic Prediction

A New Hashing based Nearest Neighbors Selection Technique for Big Datasets

Short-term Traffic Flow Prediction based on Data-Driven Knearest neighbour Nonparametric Regression

Bayes-Decisive Linear KNN with Adaptive Nearest Neighbors

Grid Point Approximation for Distributed Nonparametric Smoothing and Prediction