Abstract:As the digital world becomes the main complement to the physical world, establishing a solid line of defense against cyber attacks becomes critical and arduous. The intrusion detection systems (IDSs) based on the supervised learning method have achieved excellent performance, which requires a large amount of labeled data in the training phase. However, attacks occur much less frequently than normal behaviors, and it is difficult to obtain accurate labels. In addition, IDSs based on supervised learning cannot identify unknown attacks. At the same time, the problem that detection accuracy varies greatly with different applications is very significant in traditional unsupervised learning methods. Therefore, it is necessary to perform high-precision anomaly detection on unlabeled samples. This paper proposes a traffic anomaly detection model using K-means and Active Learning Method (ALM), which is mainly composed of a feature extraction module and an anomaly detection module. Firstly, the Pearson correlation coefficient and Light Gradient Boosting Machine (LightGBM) are used in the feature extraction module to select important features. Secondly, K-means divides the characteristic-processed traffic into normal or abnormal categories. Finally, the results of K-means are diffused through ALM, and new classification results are obtained after defuzzification, thereby improving the accuracy of anomaly detection. The latest CICDDoS2019 data set is used in the experiment. Experimental results show that the detection accuracy of the proposed model is above 90%, and the F1 score is above 95%, regardless of whether it is a binary classification of a single attack or a mixed classification of multiple attacks. Compared with three unsupervised learning methods K-means, Auto-encoder and short-term memory (LSTM) and three supervised learning methods Naive Bayes (NB), Support Vector Machine (SVM), Decision Tree (DT), the proposed model has higher classification accuracy and better generalization effect. This article is very helpful for exploring the application of unsupervised learning methods in network intrusion detection systems based on the characteristics of the data itself.

Few-Shot Class-Adaptive Anomaly Detection with Model-Agnostic Meta-Learning

A Method of Few-Shot Network Intrusion Detection Based on Meta-Learning Framework

Multimedia Traffic Anomaly Detection

Few-shot Network Anomaly Detection via Cross-network Meta-learning

A Gated Few-shot Learning Model for Anomaly Detection

MetaGAD: Meta Representation Adaptation for Few-Shot Graph Anomaly Detection

Model-agnostic generation-enhanced technology for few-shot intrusion detection

Meta-UAD: A Meta-Learning Scheme for User-level Network Traffic Anomaly Detection

Learning Hypersphere for Few-shot Anomaly Detection on Attributed Networks

MoEAD: A Parameter-efficient Model for Multi-class Anomaly Detection

Poison-Resilient Anomaly Detection: Mitigating Poisoning Attacks in Semi-Supervised Encrypted Traffic Anomaly Detection

Multitask Asynchronous Metalearning for Few-Shot Anomalous Node Detection in Dynamic Networks

Few-Shot Anomaly Detection with Adversarial Loss for Robust Feature Representations

Light-weight Unsupervised Anomaly Detection for Encrypted Malware Traffic

A Biological Immunity-Based Neuro Prototype for Few-Shot Anomaly Detection with Character Embedding

Few-Shot Bearing Anomaly Detection Based on Model-Agnostic Meta-Learning

Anomaly Intrusion Detection For Evolving Data Stream Based On Semi-Supervised Learning

From Unsupervised to Few-shot Graph Anomaly Detection: A Multi-scale Contrastive Learning Approach

Traffic Anomaly Detection Model Using K-Means and Active Learning Method

Meta pseudo labels for anomaly detection via partially observed anomalies

Network Traffic Anomaly Detection Based on Optimized Transfer Learning