Abstract:Phishing, whether through email, SMS, or malicious websites, poses a major threat to organizations by using social engineering to trick users into revealing sensitive information. It not only compromises company's data security but also incurs significant financial losses. In this paper, we investigate whether the remarkable performance of Large Language Models (LLMs) can be leveraged for particular task like text classification, particularly detecting malicious content and compare its results with state-of-the-art Deberta V3 (DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing) model. We systematically assess the potential and limitations of both approaches using comprehensive public datasets comprising diverse data sources such as email, HTML, URL, SMS, and synthetic data generation. Additionally, we demonstrate how LLMs can generate convincing phishing emails, making it harder to spot scams and evaluate the performance of both models in this context. Our study delves further into the challenges encountered by DeBERTa V3 during its training phases, fine-tuning methodology and transfer learning processes. Similarly, we examine the challenges associated with LLMs and assess their respective performance. Among our experimental approaches, the transformer-based DeBERTa method emerged as the most effective, achieving a test dataset (HuggingFace phishing dataset) recall (sensitivity) of 95.17% closely followed by GPT-4 providing a recall of 91.04%. We performed additional experiments with other datasets on the trained DeBERTa V3 model and LLMs like GPT 4 and Gemini 1.5. Based on our findings, we provide valuable insights into the effectiveness and robustness of these advanced language models, offering a detailed comparative analysis that can inform future research efforts in strengthening cybersecurity measures for detecting and mitigating phishing threats.

CANAL -- Cyber Activity News Alerting Language Model: Empirical Approach vs. Expensive LLM

SecureBERT and LLAMA 2 Empowered Control Area Network Intrusion Detection and Classification

PLLM-CS: Pre-trained Large Language Model (LLM) for cyber threat detection in satellite networks

Large Language Models for Cyber Security: A Systematic Literature Review

CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models

CS-Eval: A Comprehensive Large Language Model Benchmark for CyberSecurity

On the Uses of Large Language Models to Interpret Ambiguous Cyberattack Descriptions

Cyber Knowledge Completion Using Large Language Models

Large Language Models in Cybersecurity: State-of-the-Art

LLMs Killed the Script Kiddie: How Agents Supported by Large Language Models Change the Landscape of Network Threat Testing

CyberPal.AI: Empowering LLMs with Expert-Driven Cybersecurity Instructions

FANAL -- Financial Activity News Alerting Language Modeling Framework

Towards Characterizing Cyber Networks with Large Language Models

The Use of Large Language Models (LLM) for Cyber Threat Intelligence (CTI) in Cybercrime Forums

Cyber Situation Awareness with Active Learning for Intrusion Detection

Using Large Language Models for Cybersecurity Capture-The-Flag Challenges and Certification Questions

SecureNet: A Comparative Study of DeBERTa and Large Language Models for Phishing Detection

Evaluating the Efficacy of Large Language Models in Identifying Phishing Attempts

Next-Generation Phishing: How LLM Agents Empower Cyber Attackers