Abstract:Machine Learning (ML) techniques have been widely used in anomaly-based Intrusion Detection System (IDS) in the big data era. Although many advanced approaches are proposed recently, there are still several key limitations that should not be ignored. Firstly, data pre-processing methods have not gained sufficient attention, and they are vital to efficiency of model training especially with massive volume of collected samples. Secondly, in spite that deep learning is able to acquire more hidden interrelations from input data, it usually suffers from high complexity with numerous parameters tuned and much training time consumed. Thirdly, KDD99 data sets and their variants are leveraged by most of the literature for traffic classification, but they are proved to be outdated and inadequate for IDS evaluation. Therefore, to cope with these challenges, in this paper, we firstly propose a data pre-processing approach with under-sampling and embedded feature selection, in order to relieve the imbalance of traffic samples and extract dominant features of incoming flows. Then, we utilize LightGBM to build an traffic classification approach for IDS with better accuracy and efficiency. Finally, we evaluate our proposed approaches based on CIC-IDS2018, the data set issued in 2018 that contains comprehensive real network traffic. Extensive experiments are performed, and related results have confirmed the advantages of our proposed approaches over several other comparisons.

Robust Feature Selection for IM Applications at Early Stage Traffic Classification Using Machine Learning Algorithms

Effective Feature Selection for 5G IM Applications Traffic Classification.

Shafiq, M and Yu, X and Bashir, AK and Chaudhry, HN and Wang, D (2018)A Machine Learning Approach for Feature Selection Traffic Classification Using Security

A Machine Learning Approach for Feature Selection Traffic Classification Using Security Analysis

A New Feature Selection Method For Internet Traffic Classification Using Ml

Network Traffic Classification Via Non-Convex Multi-Task Feature Learning

Feature selection for optimizing traffic classification

Optimizing Traffic Classification Using Hybrid Feature Selection

Effective Packet Number for 5G IM WeChat Application at Early Stage Traffic Classification.

Effectiveness of Statistical Features for Early Stage Internet Traffic Identification

An Efficient Traffic Classification Scheme Using Embedded Feature Selection and LightGBM

A Statistical-Feature-based Approach to Internet Traffic Classification Using Machine Learning.

Mobile App Traffic Flow Feature Extraction and Selection for Improving Classification Robustness

On Feature Selection in Network Flow Based Traffic Sign Tracking Models

Learning Automata Based Feature Selection for Network Traffic Intrusion Detection

WeChat Text Messages Service Flow Traffic Classification Using Machine Learning Technique

Balanced Feature Selection Method for Internet Traffic Classification

Network Intrusion Detection Through Discriminative Feature Selection by Using Sparse Logistic Regression

Optimizing Feature Selection for Efficient Encrypted Traffic Classification: A Systematic Approach

Short-term Traffic Flow Forecasting Based on Feature Selection with Mutual Information