Abstract:The rapid advancement of Large Language Models (LLMs) has opened up new opportunities for leveraging artificial intelligence in various domains, including cybersecurity. As the volume and sophistication of cyber threats continue to grow, there is an increasing need for intelligent systems that can automatically detect vulnerabilities, analyze malware, and respond to attacks. In this survey, we conduct a comprehensive review of the literature on the application of LLMs in cybersecurity (LLM4Security). By comprehensively collecting over 30K relevant papers and systematically analyzing 127 papers from top security and software engineering venues, we aim to provide a holistic view of how LLMs are being used to solve diverse problems across the cybersecurity domain. Through our analysis, we identify several key findings. First, we observe that LLMs are being applied to a wide range of cybersecurity tasks, including vulnerability detection, malware analysis, network intrusion detection, and phishing detection. Second, we find that the datasets used for training and evaluating LLMs in these tasks are often limited in size and diversity, highlighting the need for more comprehensive and representative datasets. Third, we identify several promising techniques for adapting LLMs to specific cybersecurity domains, such as fine-tuning, transfer learning, and domain-specific pre-training. Finally, we discuss the main challenges and opportunities for future research in LLM4Security, including the need for more interpretable and explainable models, the importance of addressing data privacy and security concerns, and the potential for leveraging LLMs for proactive defense and threat hunting. Overall, our survey provides a comprehensive overview of the current state-of-the-art in LLM4Security and identifies several promising directions for future research.

Psychological Profiling in Cybersecurity: A Look at LLMs and Psycholinguistic Features

Towards in-situ Psychological Profiling of Cybercriminals Using Dynamically Generated Deception Environments

Large Language Models in Cybersecurity: State-of-the-Art

CS-Eval: A Comprehensive Large Language Model Benchmark for CyberSecurity

Quantifying Psychological Sophistication of Malicious Emails

Large Language Models for Cyber Security: A Systematic Literature Review

CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models

Towards Characterizing Cyber Networks with Large Language Models

AI Psychometrics: Assessing the Psychological Profiles of Large Language Models Through Psychometric Inventories

Defending Against Social Engineering Attacks in the Age of LLMs

A Survey of Large Language Models in Cybersecurity

Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models

SECURE: Benchmarking Large Language Models for Cybersecurity

Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models

When LLMs Meet Cybersecurity: A Systematic Literature Review

Forensic cyberpsychology and approaches to criminal profiling

Fundamentals of Generative Large Language Models and Perspectives in Cyber-Defense

Transforming Computer Security and Public Trust Through the Exploration of Fine-Tuning Large Language Models

Cybercrime Risk Found in Employee Behavior Big Data Using Semi-Supervised Machine Learning with Personality Theories

Ignore Me But Don't Replace Me: Utilizing Non-Linguistic Elements for Pretraining on the Cybersecurity Domain