Generating API Parameter Security Rules with LLM for API Misuse Detection

Jinghua Liu,Yi Yang,Kai Chen,Miaoqian Lin

DOI: https://doi.org/10.14722/ndss.2025.23465

2024-09-19

Abstract:In this paper, we present a new framework, named GPTAid, for automatic APSRs generation by analyzing API source code with LLM and detecting API misuse caused by incorrect parameter use. To validate the correctness of the LLM-generated APSRs, we propose an execution feedback-checking approach based on the observation that security-critical API misuse is often caused by APSRs violations, and most of them result in runtime errors. Specifically, GPTAid first uses LLM to generate raw APSRs and the Right calling code, and then generates Violation code for each raw APSR by modifying the Right calling code using LLM. Subsequently, GPTAid performs dynamic execution on each piece of Violation code and further filters out the incorrect APSRs based on runtime errors. To further generate concrete APSRs, GPTAid employs a code differential analysis to refine the filtered ones. Particularly, as the programming language is more precise than natural language, GPTAid identifies the key operations within Violation code by differential analysis, and then generates the corresponding concrete APSR based on the aforementioned operations. These concrete APSRs could be precisely interpreted into applicable detection code, which proven to be effective in API misuse detection. Implementing on the dataset containing 200 randomly selected APIs from eight popular libraries, GPTAid achieves a precision of 92.3%. Moreover, it generates 6 times more APSRs than state-of-the-art detectors on a comparison dataset of previously reported bugs and APSRs. We further evaluated GPTAid on 47 applications, 210 unknown security bugs were found potentially resulting in severe security issues (e.g., system crashes), 150 of which have been confirmed by developers after our reports.

Cryptography and Security,Software Engineering

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that when using library APIs, developers should follow API security rules to reduce the risk of API misuse. In particular, API Parameter Security Rules (APSRs) are a common type of security rule, which stipulates how to use API parameters safely and imposes restrictions on their values. Failure to comply with these rules may lead to serious security problems, such as null - pointer dereferencing and memory corruption. Manually analyzing a large number of APIs and their parameters to construct APSRs is a labor - intensive task and requires automated processing. Existing research generates APSRs from documentation and code, but due to information loss and limited analysis heuristics, the resulting APSRs are incomplete. Since large - language models (LLMs) are superior to traditional methods in code analysis and text generation without predefined heuristic rules, the authors attempt to use LLMs to address the challenges in API misuse detection. However, directly using LLMs can lead to incorrect APSRs, which may cause false vulnerability detection, and overly general APSRs that cannot generate applicable detection code, resulting in many security vulnerabilities remaining undetected.

To solve these problems, the paper proposes a new framework - GPTAid, which automatically generates APSRs by analyzing API source code and using LLMs to detect API misuse caused by improper parameter use. To verify the correctness of the APSRs generated by LLMs, the paper proposes a method based on execution - feedback checking, that is, it is observed that most security - critical API misuses are caused by violations of APSRs, and in most cases will lead to runtime errors. Specifically, GPTAid first uses LLMs to generate original APSRs and correct calling code, then generates violating code that violates each original APSRs by modifying the correct calling code. Subsequently, GPTAid dynamically executes each piece of violating code and further filters out incorrect APSRs based on runtime errors. To generate specific APSRs, GPTAid uses code - difference analysis to refine the filtered APSRs, in particular by identifying key operations in the violating code to generate corresponding specific APSRs. These specific APSRs can be precisely interpreted as applicable detection code, proving to be effective in API misuse detection.