Abstract:Anomaly detection is an important method for monitoring network traffic where is important to successfully distinguish normal traffic from abnormal traffic. For this purpose, one could use the existing classification algorithms as a part of the machine learning (ML) process. In this paper, some of the classification algorithms (Stochastic Gradient Descent (SGD), Support Vector Machines (SVM), K-Nearest Neighbor (K-NN), Gaussian Naive Bayes (GNB), Decision Tree (DT), Random Forest (RF), AdaBoost (AB)) were tested on the public UNSW-NB15 dataset. Different encoding methods and ratios of training and test data resulted in the optimal parameters classifiers. Due to the imbalanced distribution of normal and abnormal network traffic data, both standard performance scores and additional classification performance scores (F2-score, Area Under ROC Curve (AUC)) were used, that better describe the obtained results. The RF Classifier with F2-score = 97.68% and AUC score = 98.47% obtained the best results using a representative subset within the original dataset due to the shorter duration of the computations. Features in the referential dataset were reduced by 82% and selected following the structure of the NetFlow data stream. Concerning similar studies, this paper compares several algorithms for anomaly detection and selects the best one for NetFlow data streams. The F2 and AUC metric is applied, which achieves very high accuracy compared to classic metrics that do not show realistic accuracy in imbalanced datasets. Less time was spent using Label enoding (LE) with the same accuracy compared to One-hot (OH) encoding used in similar research. The novelty introduced by this paper is in the optimization of the ML process and influence of the ratio of data for learning and testing, different encoding methods of categorical features, and feature reduction on the NetFlow data streams.

Network anomaly detection and classification via opportunistic sampling

An Efficient Sampling And Classification Approach For Flow Detection In Sdn-Based Big Data Centers

A signal analysis of network traffic anomalies

Network Anomaly Detection: A Survey and Comparative Analysis of Stochastic and Deterministic Methods

Impact of packet sampling on anomaly detection metrics

A State-Space Approach for Optimal Traffic Monitoring via Network Flow Sampling

Anomaly Detections in Internet traffic Using Empirical Measures

A network anomaly detection algorithm based on semi-supervised learning and adaptive multiclass balancing

Anomaly detection system for network transport with machine learning approach

: Towards Fine-Grained Unknown Class Detection Against the Open-Set Attack Spectrum with Variable Legitimate Traffic

Survey on Incremental Approaches for Network Anomaly Detection

Distributed Online One-Class Support Vector Machine for Anomaly Detection over Networks

Detection of Sparse Anomalies in High-Dimensional Network Telescope Signals

Network Sampling: From Static to Streaming Graphs

Unsupervised Abnormal Traffic Detection through Topological Flow Analysis

An empirical study for the traffic flow rate prediction-based anomaly detection in software-defined networking: a challenging overview

Traffic Classification - Towards Accurate Real Time Network Applications

Borderline SMOTE Algorithm and Feature Selection-Based Network Anomalies Detection Strategy

Anomaly detection in NetFlow network traffic using supervised machine learning algorithms

Accurate and Efficient Traffic Monitoring Using Adaptive Non-Linear Sampling Method