Abstract:The commonly used method for network intrusion is based on pattern matching systems, but these systems do not have high detection accuracy when facing large-scale high-speed network traffic environments. In addition, when facing a wide variety of network attacks, relying solely on a certain network security technology is difficult to ensure network security. Therefore, a distributed network intrusion prevention system based on the Spark framework and dynamic information security theory model was innovatively proposed to improve the detection efficiency of network intrusion and solve these issues. The architecture and functional structure of the network intrusion prevention system were constructed by improving the random forest algorithm and Spark. In addition, the dynamic network intrusion prevention system was designed on the basis of the Policy Protection Detection Response (P2DR) model and the Protection Detection Reaction Recovery (PDRR) model to cope with the diverse network attacks and make up for the lack of dynamic defense based on improved forest algorithm and Spark. The test results showed that the network intrusion prevention system based on the improved random forest algorithm and Spark had a maximum detection time of 13,593 ms, a minimum detection time of 13,318 ms, and an average detection time of 13,468 ms when the data were 6000 pieces. The average success rate of the dynamic network intrusion prevention system based on the dynamic information security theory model was 87.72%, the average detection rate was 71.68%, and the average false alarm rate was 17.23%. The F1 values corresponding to the improved random forest algorithm under 7 different attack types of data were 0.985, 0.983, 0.876, 0.843, 0.797, 0.983, and 0.890, respectively, which were significantly better than the comparison algorithm. It can be seen that the dynamic network intrusion prevention system based on the improved random forest algorithm, Spark, and the dynamic information security theoretical model, has good performance and can effectively detect network intrusions, providing technical support for solving network intrusion problems in reality. The contribution of the research is reflected in the improvement of the detection performance of network intrusion detection systems and the reduction of detection time and false alarm rates.

Real-time Distributed-Random-Forest-Based Network Intrusion Detection System Using Apache Spark

NIDS: Neural Network Based Intrusion Detection System

A Deep Random Forest Model on Spark for Network Intrusion Detection

Design of distributed network intrusion prevention system based on Spark and P2DR models

Research on Data Mining-based Intrusion Detection in Real-time Environment

Design and application of real-time network abnormal traffic detection system based on Spark Streaming

Fuzzy Local Information and Bhattacharya-Based C-Means Clustering and Optimized Deep Learning in Spark Framework for Intrusion Detection

An accurate IoT Intrusion Detection Framework using Apache Spark

NIDS: Random Forest Based Novel Network Intrusion Detection System for Enhanced Cybersecurity in VANET's

Towards Near-Real-Time Intrusion Detection for IoT Devices using Supervised Learning and Apache Spark

Big data analysis and distributed deep learning for next-generation intrusion detection system optimization

A Speculative Parallel Intrusion Prevention System Based on Apache Spark

Apache Spark and Deep Learning Models for High-Performance Network Intrusion Detection Using CSE-CIC-IDS2018

An Efficient Network Intrusion Detection Approach Based on Deep Learning

A Distributed Network Intrusion Detection System for Distributed Denial of Service Attacks in Vehicular Ad Hoc Network

Machine-Learning-Based Online Distributed Denial-of-Service Attack Detection Using Spark Streaming

Performance Evaluation of Apache Spark MLlib Algorithms on an Intrusion Detection Dataset

Real-Time Network Intrusion Detection System Based on Deep Learning

Work-in-Progress: Towards Real-Time IDS Via RNN and Programmable Switches Co-Designed Approach

A distributed platform for intrusion detection system using data stream mining in a big data environment

Internet Traffic Analysis Using Community Detection and Apache Spark