An Explainable Transformer-based Model for Phishing Email Detection: A Large Language Model Approach

Mohammad Amaz Uddin,Iqbal H. Sarker

2024-02-21

Abstract:Phishing email is a serious cyber threat that tries to deceive users by sending false emails with the intention of stealing confidential information or causing financial harm. Attackers, often posing as trustworthy entities, exploit technological advancements and sophistication to make detection and prevention of phishing more challenging. Despite extensive academic research, phishing detection remains an ongoing and formidable challenge in the cybersecurity landscape. Large Language Models (LLMs) and Masked Language Models (MLMs) possess immense potential to offer innovative solutions to address long-standing challenges. In this research paper, we present an optimized, fine-tuned transformer-based DistilBERT model designed for the detection of phishing emails. In the detection process, we work with a phishing email dataset and utilize the preprocessing techniques to clean and solve the imbalance class issues. Through our experiments, we found that our model effectively achieves high accuracy, demonstrating its capability to perform well. Finally, we demonstrate our fine-tuned model using Explainable-AI (XAI) techniques such as Local Interpretable Model-Agnostic Explanations (LIME) and Transformer Interpret to explain how our model makes predictions in the context of text classification for phishing emails.

Artificial Intelligence,Machine Learning,Cryptography and Security

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the challenges in **phishing email detection**. Specifically, the paper focuses on the following aspects:

1. **Severity of phishing emails**: Phishing emails are a serious cyber - threat. Attackers attempt to steal users' confidential information or cause financial losses by sending fake emails. With the progress and increasing complexity of technology, the detection and prevention of phishing emails have become more difficult.

2. **Limitations of existing methods**: Although much research has been devoted to phishing email detection, existing methods still have some shortcomings, especially in dealing with unbalanced data sets and explaining model predictions.

3. **Taking advantage of large - language models**: Large - language models (LLMs) and masked - language models (MLMs) have great potential and can provide innovative solutions to long - standing challenges. The paper proposes an optimized and fine - tuned DistilBERT model for phishing email detection.

4. **Model interpretability**: To improve the transparency and credibility of the model, the paper uses two explainable artificial intelligence (XAI) techniques - Local Interpretable Model - agnostic Explanations (LIME) and Transformer Interpret - to explain how the model makes predictions.

### Main contributions of the paper
- **Proposes a Transformer - based phishing email detection model**, which is a fine - tuned version of DistilBERT.
- **Utilizes two different XAI techniques, LIME and Transformer Interpret**, explores the interpretability of the fine - tuned model, and visualizes the percentage scores of these techniques in prediction.
- **Demonstrates the performance of the model on unbalanced and balanced data sets**.

Through these contributions, the paper not only improves the accuracy of phishing email detection but also enhances the transparency and interpretability of the model, which is crucial for trust and adoption in practical applications.