Abstract:Due to the proliferation of large language models (LLMs) and their widespread use in applications such as ChatGPT, there has been a significant increase in interest in AI over the past year. Multiple researchers have raised the question: how will AI be applied and in what areas? Programming, including the generation, interpretation, analysis, and documentation of static program code based on promptsis one of the most promising fields. With the GPT API, we have explored a new aspect of this: static analysis of the source code of front-end applications at the endpoints of the data path. Our focus was the detection of the CWE-653 vulnerability—inadequately isolated sensitive code segments that could lead to unauthorized access or data leakage. This type of vulnerability detection consists of the detection of code segments dealing with sensitive data and the categorization of the isolation and protection levels of those segments that were previously not feasible without human intervention. However, we believed that the interpretive capabilities of GPT models could be explored to create a set of prompts to detect these cases on a file-by-file basis for the applications under study, and the efficiency of the method could pave the way for additional analysis tasks that were previously unavailable for automation. In the introduction to our paper, we characterize in detail the problem space of vulnerability and weakness detection, the challenges of the domain, and the advances that have been achieved in similarly complex areas using GPT or other LLMs. Then, we present our methodology, which includes our classification of sensitive data and protection levels. This is followed by the process of preprocessing, analyzing, and evaluating static code. This was achieved through a series of GPT prompts containing parts of static source code, utilizing few-shot examples and chain-of-thought techniques that detected sensitive code segments and mapped the complex code base into manageable JSON structures.Finally, we present our findings and evaluation of the open source project analysis, comparing the results of the GPT-based pipelines with manual evaluations, highlighting that the field yields a high research value. The results show a vulnerability detection rate for this particular type of model of 88.76%, among others.

A Data-Mining Based Study of Security Vulnerability Types and Their Mitigation in Different Languages

MegaVul: A C/C++ Vulnerability Dataset with Comprehensive Code Representations.

Exploring the Security Awareness of the Python and JavaScript Open Source Communities

An Empirical Study of Developers' Discussions about Security Challenges of Different Programming Languages

A Historical and Statistical Studyof the Software Vulnerability Landscape

What Security Questions Do Developers Ask? A Large-Scale Study of Stack Overflow Posts.

Causative Insights into Open Source Software Security using Large Language Code Embeddings and Semantic Vulnerability Graph

An empirical study of text-based machine learning models for vulnerability detection

Vulnerability discovery based on source code patch commit mining: a systematic literature review

How to build a vulnerability benchmark to overcome cyber security attacks

A Large-Scale Empirical Study on Vulnerability Distribution Within Projects and the Lessons Learned

Insights and Current Gaps in Open-Source LLM Vulnerability Scanners: A Comparative Analysis

SQVDT: A Scalable Quantitative Vulnerability Detection Technique for Source Code Security Assessment.

Explaining the Contributing Factors for Vulnerability Detection in Machine Learning

Vulnerability Scanners for Ethereum Smart Contracts: A Large-Scale Study

A New Approach to Web Application Security: Utilizing GPT Language Models for Source Code Inspection

A ground-truth dataset of real security patches

Software Vulnerability Prediction in Low-Resource Languages: An Empirical Study of CodeBERT and ChatGPT

What are developers talking about information security? A large-scale study using semantic analysis of Q&A posts

A Mining Approach to Obtain the Software Vulnerability Characteristics