Abstract:The k-nearest neighbor algorithm has been widely used in network anomaly detection works, but its query efficiency decreases significantly when the number of samples and feature dimensions increase. To meet the demand for real-time detection, an accurate and timely anomaly detection solution is particularly important. This paper proposes a fast anomaly traffic detection method based on the constrained k-nearest neighbor (CKNN) algorithm. The method uses equilibrium modified k-means and randomized incremental method to optimize the ball tree construction scheme. Specifically, randomized incremental method is used to solve the minimum coverage ball, which optimizes the selection process of the centeroid and radius while reducing the depth of the ball tree. And the equilibrium modified k-means method replaces the original k-center principle used in the division of subtrees, which solves the problem of unbalanced search binary tree division. Meanwhile, by reducing the required number of backtracking to search the K nearest neighbors, which reduces the classification time overhead of the algorithm. We validate the effectiveness of the method on several benchmark datasets. The experimental results show that the CKNN maintains a higher query rate without loss of detection accuracy when dealing with high-dimensional, massive sample data compared with the traditional KNN algorithm. And with the growth ratio of up to 99.68% for some samples, our method also exhibits higher detection accuracy and less time consumption compared with other machine learning algorithms.

Fast kNN Graph Construction with Locality Sensitive Hashing.

Fast Nearest Neighbor Search Based on Approximate K-Nn Graph

Fast Approximate K NN Graph Construction for High Dimensional Data Via Recursive Lanczos Bisection

Scalable $k$-NN graph construction.

Fast K-Means Based on KNN Graph

Efficient kNN Algorithm Based on Graph Sparse Reconstruction.

Fast Approximate Nearest Neighbor Search Via K-Diverse Nearest Neighbor Graph.

Approximate K-Nn Graph Construction: A Generic Online Approach

Scalable K-Nn Graph Construction for Visual Descriptors

Fast Graph Similarity Search via Locality Sensitive Hashing.

K-Nn Graph Construction: a Generic Online Approach

Scalable Nearest Neighbor Search based on kNN Graph

Large-Scale Approximate k-NN Graph Construction on GPU

Fast K-Nn Graph Construction by GPU Based NN-Descent

Preserving-Ignoring Transformation Based Index for Approximate k Nearest Neighbor Search

A fast anomaly network traffic detection method based on the constrained k-nearest neighbor

Efficient k NN Search in Public Transportation Networks

Learning Efficient Hash Codes for Fast Graph-Based Data Similarity Retrieval.

A New Hashing based Nearest Neighbors Selection Technique for Big Datasets

Fast $k$-NNG construction with GPU-based quick multi-select

Dynamic NN-Descent: an Efficient K-Nn Graph Construction Method