Abstract:The k-nearest neighbor algorithm has been widely used in network anomaly detection works, but its query efficiency decreases significantly when the number of samples and feature dimensions increase. To meet the demand for real-time detection, an accurate and timely anomaly detection solution is particularly important. This paper proposes a fast anomaly traffic detection method based on the constrained k-nearest neighbor (CKNN) algorithm. The method uses equilibrium modified k-means and randomized incremental method to optimize the ball tree construction scheme. Specifically, randomized incremental method is used to solve the minimum coverage ball, which optimizes the selection process of the centeroid and radius while reducing the depth of the ball tree. And the equilibrium modified k-means method replaces the original k-center principle used in the division of subtrees, which solves the problem of unbalanced search binary tree division. Meanwhile, by reducing the required number of backtracking to search the K nearest neighbors, which reduces the classification time overhead of the algorithm. We validate the effectiveness of the method on several benchmark datasets. The experimental results show that the CKNN maintains a higher query rate without loss of detection accuracy when dealing with high-dimensional, massive sample data compared with the traditional KNN algorithm. And with the growth ratio of up to 99.68% for some samples, our method also exhibits higher detection accuracy and less time consumption compared with other machine learning algorithms.

The active leaning-based nearest neighbor mean distance novelty detection for large data set

Privacy preserving and fast decision for novelty detection using support vector data description

Active Learning Based Support Vector Data Description Method for Robust Novelty Detection

Anomaly detection based on local nearest neighbor distance descriptor in crowded scenes.

Nearest Neighbor Classifier Embedded Network for Active Learning

Non-Adversarial Novelty Detection with Generative Latent Nearest Neighbors

Learning to Detect Interesting Anomalies

Active and Passive Nearest Neighbor Algorithm: A Newly-Developed Supervised Classifier

A fast anomaly network traffic detection method based on the constrained k-nearest neighbor

Anomaly detection based on Nearest Neighbor search with Locality-Sensitive B-tree.

NDOD: an Efficient Neighboring Dependent Outlier Detector for Bias Distributed Large Datasets

A Novel Pseudo Nearest Neighbor Classification Method Using Local Harmonic Mean Distance

Nearest Neighbor Classifier with Margin Penalty for Active Learning

A Local Mean Representation-based K-Nearest Neighbor Classifier

AutoMS: Automatic Model Selection for Novelty Detection with Error Rate Control

Algorithms that Defy the Gravity of Learning Curve

Incremental Kernel Null Space Discriminant Analysis for Novelty Detection

Novel Class Discovery for Long-tailed Recognition

A hybrid real-valued negative selection algorithm with variable-sized detectors and the k-nearest neighbors algorithm

k-NNN: Nearest Neighbors of Neighbors for Anomaly Detection

Efficient Data-aware Distance Comparison Operations for High-Dimensional Approximate Nearest Neighbor Search