Abstract:Due to the insidious characteristics of network intrusion behaviors, developing an efficient intrusion detection system is still a big challenge, especially in the era of big data where the number of traffic and the dimension of each traffic feature are high. Because of the shortcomings of traditional common machine learning algorithms in network intrusion detection, such as insufficient accuracy, a network intrusion detection system based on LightGBM and autoencoder (AE) is proposed. The LightGBM-AE model proposed in this paper includes three steps: data preprocessing, feature selection, and classification. The LightGBM-AE model adopts the LightGBM algorithm for feature selection, and then uses an autoencoder for training and detection. When a set of data containing network intrusion behaviors are inputted into an autoencoder, there is a large reconstruction error between the original input data and the reconstructed data obtained by the autoencoder, which provides a basis for intrusion detection. According to the reconstruction error, an appropriate threshold is set to distinguish symmetrically between normal behavior and attack behavior. The experiment is carried out on the NSL-KDD dataset and implemented using Pytorch. In addition to autoencoder, variational autoencoder (VAE) and denoising autoencoder (DAE) are also used for intrusion detection and are compared with existing machine learning algorithms such as Decision Tree, Random Forest, KNN, GBDT, and XGBoost. The evaluation is carried out through classification evaluation indexes such as accuracy, precision, recall, F1-score. The experimental results show that the method can efficiently separate the attack behavior from normal behavior according to the reconstruction error. Compared with other methods, the effectiveness and superiority of this method are verified.

Leaky Training: A Robustness Improvement Scheme for Autoencoder Based Network Intrusion Detection System under Online Learning

Deep Learning for Network Anomaly Detection under Data Contamination: Evaluating Robustness and Mitigating Performance Degradation

Enhancing network intrusion detection by lifelong active online learning

Unknown Security Attack Detection of Industrial Control System by Deep Learning

Analyzing Autoencoder-Based Intrusion Detection System Performance

Reinforcement Learning Meets Network Intrusion Detection: A Transferable and Adaptable Framework for Anomaly Behavior Identification

Network Intrusion Detection Based on Supervised Adversarial Variational Auto-Encoder With Regularization

ARCADE: Adversarially Regularized Convolutional Autoencoder for Network Anomaly Detection

A Deep Learning Model for Network Intrusion Detection with Imbalanced Data

A Comparative Study of Off-Line Deep Learning Based Network Intrusion Detection

A network security situation assessment method based on adversarial deep learning

A network intrusion detection framework on sparse deep denoising auto-encoder for dimensionality reduction

A Self-supervised Adversarial Learning Approach for Network Intrusion Detection System.

Training Strategies for Autoencoder-based Detection of False Data Injection Attacks

Improving the Classification Effectiveness of Intrusion Detection by Using Improved Conditional Variational AutoEncoder and Deep Neural Network

Addressing the class imbalance problem in network intrusion detection systems using data resampling and deep learning

An Enhanced AI-Based Network Intrusion Detection System Using Generative Adversarial Networks

Network Anomaly Intrusion Detection Based on Deep Learning Approach

An Efficient Intrusion Detection Method Based on LightGBM and Autoencoder

Deep Adversarial Learning in Intrusion Detection: A Data Augmentation Enhanced Framework

A Network Traffic Intrusion Detection Method for Industrial Control Systems Based on Deep Learning