Abstract:Due to the insidious characteristics of network intrusion behaviors, developing an efficient intrusion detection system is still a big challenge, especially in the era of big data where the number of traffic and the dimension of each traffic feature are high. Because of the shortcomings of traditional common machine learning algorithms in network intrusion detection, such as insufficient accuracy, a network intrusion detection system based on LightGBM and autoencoder (AE) is proposed. The LightGBM-AE model proposed in this paper includes three steps: data preprocessing, feature selection, and classification. The LightGBM-AE model adopts the LightGBM algorithm for feature selection, and then uses an autoencoder for training and detection. When a set of data containing network intrusion behaviors are inputted into an autoencoder, there is a large reconstruction error between the original input data and the reconstructed data obtained by the autoencoder, which provides a basis for intrusion detection. According to the reconstruction error, an appropriate threshold is set to distinguish symmetrically between normal behavior and attack behavior. The experiment is carried out on the NSL-KDD dataset and implemented using Pytorch. In addition to autoencoder, variational autoencoder (VAE) and denoising autoencoder (DAE) are also used for intrusion detection and are compared with existing machine learning algorithms such as Decision Tree, Random Forest, KNN, GBDT, and XGBoost. The evaluation is carried out through classification evaluation indexes such as accuracy, precision, recall, F1-score. The experimental results show that the method can efficiently separate the attack behavior from normal behavior according to the reconstruction error. Compared with other methods, the effectiveness and superiority of this method are verified.

Malicious Behavior Catcher: an Intrusion Detection System Based on VAE

A Mixed Intrusion Detection System Utilizing K-means and Extreme Gradient Boosting

Improving the Classification Effectiveness of Intrusion Detection by Using Improved Conditional Variational AutoEncoder and Deep Neural Network

Intrusion Detection System Using Voting-Based Neural Network

A Lightweight Unsupervised Intrusion Detection Model Based on Variational Auto-Encoder

An Efficient Intrusion Detection Method Based on LightGBM and Autoencoder

Network Intrusion Detection Based on Supervised Adversarial Variational Auto-Encoder With Regularization

Trustworthy Intrusion Detection: Confidence Estimation Using Latent Space

A Deep Random Forest Model on Spark for Network Intrusion Detection

Efficient Deep CNN-BiLSTM Model for Network Intrusion Detection

An Incremental Majority Voting Approach for Intrusion Detection System Based on Machine Learning

Toward Effective Intrusion Detection Using Log-Cosh Conditional Variational Autoencoder

An Explainable Ensemble-based Intrusion Detection System for Software-Defined Vehicle Ad-hoc Networks

A-CAVE: Network abnormal traffic detection algorithm based on variational autoencoder

A Network Traffic Intrusion Detection Method for Industrial Control Systems Based on Deep Learning

Detecting Unbalanced Network Traffic Intrusions With Deep Learning

Intrusion Detection System After Data Augmentation Schemes Based on the VAE and CVAE

Learning to Detect: A Data-driven Approach for Network Intrusion Detection

A Deep Learning Model for Network Intrusion Detection with Imbalanced Data

A Scalable Network Intrusion Detection System using Bi-LSTM and CNN

A Network Intrusion Detection Model Based on BiLSTM with Multi-Head Attention Mechanism