Automated Software Vulnerability Patching using Large Language Models

Yu Nong,Haoran Yang,Long Cheng,Hongxin Hu,Haipeng Cai

2024-08-24

Abstract:Timely and effective vulnerability patching is essential for cybersecurity defense, for which various approaches have been proposed yet still struggle to generate valid and correct patches for real-world vulnerabilities. In this paper, we leverage the power and merits of pre-trained large language models (LLMs) to enable automated vulnerability patching using no test input/exploit evidence and without model training/fine-tuning. To elicit LLMs to effectively reason about vulnerable code behaviors, which is essential for quality patch generation, we introduce adaptive prompting on LLMs and instantiate the methodology as LLMPATCH, an automated LLM-based patching system. Our evaluation of LLMPATCH on real-world vulnerable code including zeroday vulnerabilities demonstrates its superior performance to both existing prompting methods and state-of-the-art non-LLM-based techniques (by 98.9% and 65.4% in F1 over the best baseline performance). LLMPATCH has also successfully patched 7 out of 11 zero-day vulnerabilities, including 2 that none of the four baselines compared were able to.

Cryptography and Security,Software Engineering

What problem does this paper attempt to address?

The problem this paper attempts to address is the challenge of automated software vulnerability patching in practical applications. Specifically, existing methods, while capable of generating patch code, often fail to produce effective and correct patches when dealing with real-world vulnerabilities, especially in the absence of test inputs or exploit evidence. Additionally, deep learning-based methods require a large amount of labeled data for training, which is difficult to obtain in reality, leading to poor performance when handling unseen code.

To overcome these issues, the paper proposes an automated vulnerability patching method based on pre-trained large language models (LLMs) called LLMPATCH. This method introduces an adaptive prompting strategy that can effectively analyze vulnerability behavior and generate high-quality patches without any test inputs or model training/fine-tuning. The paper experimentally validates the superior performance of LLMPATCH in handling real-world vulnerabilities, including zero-day vulnerabilities.