Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation

Jiawei Liu,Chunqiu Steven Xia,Yuyao Wang,Lingming Zhang

2023-10-31

Abstract:Program synthesis has been long studied with recent approaches focused on directly using the power of Large Language Models (LLMs) to generate code. Programming benchmarks, with curated synthesis problems and test-cases, are used to measure the performance of various LLMs on code synthesis. However, these test-cases can be limited in both quantity and quality for fully assessing the functional correctness of the generated code. Such limitation in the existing benchmarks begs the following question: In the era of LLMs, is the code generated really correct? To answer this, we propose EvalPlus -- a code synthesis evaluation framework to rigorously benchmark the functional correctness of LLM-synthesized code. EvalPlus augments a given evaluation dataset with large amounts of test-cases newly produced by an automatic test input generator, powered by both LLM- and mutation-based strategies. While EvalPlus is general, we extend the test-cases of the popular HumanEval benchmark by 80x to build HumanEval+. Our extensive evaluation across 26 popular LLMs (e.g., GPT-4 and ChatGPT) demonstrates that HumanEval+ is able to catch significant amounts of previously undetected wrong code synthesized by LLMs, reducing the pass@k by up-to 19.3-28.9%. We also surprisingly found that test insufficiency can lead to mis-ranking. For example, both WizardCoder-CodeLlama and Phind-CodeLlama now outperform ChatGPT on HumanEval+, while none of them could on HumanEval. Our work not only indicates that prior popular code synthesis evaluation results do not accurately reflect the true performance of LLMs for code synthesis, but also opens up a new direction to improve such programming benchmarks through automated testing. We have open-sourced our tools, enhanced datasets as well as all LLM-generated code at <a class="link-external link-https" href="https://github.com/evalplus/evalplus" rel="external noopener nofollow">this https URL</a> to facilitate and accelerate future LLM-for-code research.

Software Engineering,Computation and Language,Machine Learning

What problem does this paper attempt to address?

The paper primarily explores the issue of evaluating the correctness of code generated by large language models (LLMs). Specifically, existing programming benchmarks (such as HUMAN EVAL) include some carefully designed synthetic problems and test cases to measure the performance of different LLMs in code generation tasks. However, these test cases have limitations in both quantity and quality, making it impossible to comprehensively assess the functional correctness of the generated code. Therefore, the paper proposes a code synthesis evaluation framework called EvalPlus, which is used for rigorous benchmarking of the functional correctness of code generated by LLMs.

EvalPlus enhances existing evaluation datasets in the following ways:
1. **Automatic Test Input Generation**: Utilizes LLMs (such as ChatGPT) to generate high-quality seed inputs and combines mutation testing methods to generate a large number of new test inputs.
2. **Test Suite Reduction**: Employs a greedy set cover algorithm to select the most valuable test cases, reducing the cost of test execution while maintaining the same test effectiveness.
3. **Program Input Contracts**: Introduces programming contract annotations to ensure that test inputs conform to the expected format, avoiding uncertain behavior caused by invalid inputs.

Through extensive evaluation on 26 popular LLMs, the study found that when using HUMAN EVAL+ (the extended benchmark), the `pass@k` results of almost all models significantly declined, indicating that existing evaluation methods are insufficient to detect errors in code generated by LLMs. Additionally, the study discovered errors in some ground truth solutions within the benchmarks, further questioning the quality of existing code synthesis benchmarks. In summary, the paper emphasizes the importance of improving code synthesis evaluation methods and provides open-source tools and enhanced datasets to facilitate future research.