Hotfixing Large Language Models for Code

Zhou Yang,David Lo

2024-10-13

Abstract:Large Language Models for Code (LLM4Code) have become an integral part of developers' workflows, assisting with tasks such as code completion and generation. However, these models are found to exhibit undesired behaviors after their release, like generating buggy code, due to their extensive training on vast amounts of source code that contain such buggy code. The training data (usually coming from open-source software) keeps evolving, e.g., developers fix the buggy code. However, adapting such evolution to mitigate LLM4Code's undesired behaviors is non-trivial, as retraining models on the updated dataset usually takes much time and resources. This motivates us to propose the concept of hotfixing LLM4Code, mitigating LLM4Code's undesired behaviors effectively and efficiently with minimal negative effects. This paper mainly focuses on hotfixing LLM4Code to make them generate less buggy code and more fixed code. We begin by demonstrating that models from the popular CodeGen family frequently generate buggy code. Then, we define three learning objectives in hotfixing and design multiple loss functions for each objective: (1) learn the desired behaviors, (2) unlearn the undesired behaviors, and (3) retain knowledge of other code. We evaluate four different fine-tuning techniques for hotfixing the models and gain the following insights. Optimizing these three learning goals together, using LoRA (low-rank adaptation), effectively influences the model's behavior. Specifically, it increases the generation of fixed code by up to 108.42% and decreases the generation of buggy code by up to 50.47%. Statistical tests confirm that hotfixing does not significantly affect the models' functional correctness on the HumanEval benchmark. Additionally, to evaluate the generalizability of hotfixing by reducing the exposure of email addresses by 99.30%.

Software Engineering

What problem does this paper attempt to address?

### What problems does this paper attempt to solve?

This paper aims to solve the problem of the bad behavior of large - language models (LLM4Code) in code - generation tasks. Specifically, these models may exhibit some undesired behaviors after being released, such as generating buggy code. Since these models are trained on a large amount of open - source code, and this open - source code may contain unfixed errors, the models have learned to generate similar error - prone code. Moreover, as open - source code repositories are continuously updated and fixed, retraining the models to adapt to these changes is both time - consuming and resource - intensive.

To solve these problems, the authors propose the concept of "hotfixing", that is, fine - tuning LLM4Code in an efficient and minimally - invasive way to reduce the probability of generating error - prone code and improve the ability to generate fixed code. The goals of hotfixing are:

1. **Learn desired behaviors**: Make the model learn to generate correct, fixed code.
2. **Unlearn undesired behaviors**: Make the model avoid generating error - prone code.
3. **Retain knowledge of other code**: Ensure that the model's performance in other aspects (parts unrelated to error - fixing) is not affected.

To achieve these goals, the authors design multiple loss functions and use parameter - efficient fine - tuning methods (PEFT) to update the model. Experimental results show that this method can significantly reduce the proportion of error - prone code generated by the model and increase the proportion of fixed - code generation, without having a significant negative impact on the model's functional correctness.

### Specific problem description

- **Problem background**: LLM4Code has exhibited the bad behavior of generating error - prone code in code - generation tasks, which may be due to their being trained on open - source code containing errors.
- **Problem impact**: These bad behaviors may cause developers to introduce errors when using these models, affecting code quality and development efficiency.
- **Solution**: A method called "hotfixing" is proposed. By fine - tuning the model, the probability of generating error - prone code is reduced, and the ability to generate fixed code is improved, while minimizing the impact on other functions of the model.

### Specific goals of hotfixing

1. **Generate less error - prone code**: By learning the fixed code, make the model able to generate less error - prone code.
2. **Generate more fixed code**: Make the model able to generate more fixed code.
3. **Maintain the performance of other code**: Ensure that the model's performance in other code - generation tasks is not affected.

Through these measures, hotfixing can quickly and effectively improve the model's performance in code - generation tasks without affecting the overall performance of the model.