Machine-Learning-Based Online Distributed Denial-of-Service Attack Detection Using Spark Streaming
Baojun Zhou,Jie Li,Jinsong Wu,Song Guo,Yu Gu,Zhetao Li
DOI: https://doi.org/10.1109/icc.2018.8422327
2018-01-01
Abstract:In order to cope with the increasing number of cyber attacks, network operators must monitor the whole network situations in real time. Traditional network monitoring method that usually works on a single machine, however, is no longer suitable for the huge traffic data nowadays due to its poor processing ability. In this paper, we propose a machine-learning based online Internet traffic monitoring system using Spark Streaming, a stream- processing-based big data framework, to detect DDoS attacks in real time. The system consists of three parts, collector, messaging system and stream processor. We use a correlation-based feature selection method and choose 4 most necessary network features in our machine- learning-based DDoS detection algorithm. We verify the result of feature selection method by a comparative experiment and compare the detection accuracy of 3 machine learning methods - Naïve Bayes, Logistic Regression and Decision Tree. Finally, we conduct experiments in a cluster with the standalone mode, showing that our system can detect 3 typical DDoS attacks - TCP flooding, UDP flooding and ICMP flooding at the accuracy of more than 99.3%. It also shows the system performs well even for large Internet traffic.