A Transformer-Based Framework for Payload Malware Detection and Classification

Kyle Stein,Arash Mahyari,Guillermo Francia III,Eman El-Sheikh

2024-03-27

Abstract:As malicious cyber threats become more sophisticated in breaching computer networks, the need for effective intrusion detection systems (IDSs) becomes crucial. Techniques such as Deep Packet Inspection (DPI) have been introduced to allow IDSs analyze the content of network packets, providing more context for identifying potential threats. IDSs traditionally rely on using anomaly-based and signature-based detection techniques to detect unrecognized and suspicious activity. Deep learning techniques have shown great potential in DPI for IDSs due to their efficiency in learning intricate patterns from the packet content being transmitted through the network. In this paper, we propose a revolutionary DPI algorithm based on transformers adapted for the purpose of detecting malicious traffic with a classifier head. Transformers learn the complex content of sequence data and generalize them well to similar scenarios thanks to their self-attention mechanism. Our proposed method uses the raw payload bytes that represent the packet contents and is deployed as man-in-the-middle. The payload bytes are used to detect malicious packets and classify their types. Experimental results on the UNSW-NB15 and CIC-IOT23 datasets demonstrate that our transformer-based model is effective in distinguishing malicious from benign traffic in the test dataset, attaining an average accuracy of 79\% using binary classification and 72\% on the multi-classification experiment, both using solely payload bytes.

Cryptography and Security,Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

The paper aims to address the problem of malware detection and classification in Intrusion Detection Systems (IDS). Specifically, the researchers propose a Transformer-based Deep Packet Inspection (DPI) algorithm to analyze the content of network packets to identify malicious traffic and further classify this malicious traffic.

The main contributions of the paper include:
1. **Proposing a new DPI algorithm**: This algorithm is based on the Transformer model and can capture complex patterns and dependencies in the raw payload bytes of network packets through the self-attention mechanism.
2. **Effectively handling the challenges posed by encrypted traffic**: Although encrypted traffic hides useful information in packets, this method performs well on unencrypted data, effectively distinguishing between malicious and benign traffic.
3. **Experimental validation**: The research used two well-known and widely used datasets, UNSW-NB15 and CIC-IOT23, for experimental validation. The results show that in binary classification tasks, the proposed model achieved an average accuracy of 79%, and in multi-class classification tasks, the accuracy was 72%.

Additionally, the paper discusses the impact of encrypted traffic on malware detection, pointing out that encryption algorithms like AES can effectively hide patterns in packets, while some encryption algorithms may not be strong enough and could still reveal characteristics of malware. Finally, by comparing the performance of different models (including 1D Convolutional Neural Networks (1D-CNN), 2D Convolutional Neural Networks (2D-CNN), and Long Short-Term Memory Networks (LSTM)), it is demonstrated that the proposed Transformer-based method has higher accuracy and robustness in malware detection.