Abstract:With the increasing utilization of large language models such as ChatGPT during software development, it has become crucial to verify the quality of code content it generates. Recent studies proposed utilizing ChatGPT as both a developer and tester for multi-agent collaborative software development. The multi-agent collaboration empowers ChatGPT to produce test reports for its generated code, enabling it to self-verify the code content and fix bugs based on these reports. However, these studies did not assess the effectiveness of the generated test reports in validating the code. Therefore, we conduct a comprehensive empirical investigation to evaluate ChatGPT's self-verification capability in code generation, code completion, and program repair. We request ChatGPT to (1) generate correct code and then self-verify its correctness; (2) complete code without vulnerabilities and then self-verify for the presence of vulnerabilities; and (3) repair buggy code and then self-verify whether the bugs are resolved. Our findings on two code generation datasets, one code completion dataset, and two program repair datasets reveal the following observations: (1) ChatGPT often erroneously predicts its generated incorrect code as correct. (2) The self-contradictory hallucinations in ChatGPT's behavior arise. (3) The self-verification capability of ChatGPT can be enhanced by asking the guiding question, which queries whether ChatGPT agrees with assertions about incorrectly generated or repaired code and vulnerabilities in completed code. (4) Using test reports generated by ChatGPT can identify more vulnerabilities in completed code, but the explanations for incorrectly generated code and failed repairs are mostly inaccurate in the test reports. Based on these findings, we provide implications for further research or development using ChatGPT.

Investigating the Utility of ChatGPT in the Issue Tracking System: An Exploratory Study

Why Do Developers Engage with ChatGPT in Issue-Tracker? Investigating Usage and Reliance on ChatGPT-Generated Code

The Potential Use of ChatGPT for Debugging and Bug Fixing

ChatGPT: A Study on its Utility for Ubiquitous Software Engineering Tasks

"Will I be replaced?" Assessing ChatGPT's effect on software development and programmer perceptions of AI tools

An empirical study on developers' shared conversations with ChatGPT in GitHub pull requests and issues

ChatGPT as a Software Development Bot: A Project-based Study

An Empirical Study on Developers Shared Conversations with ChatGPT in GitHub Pull Requests and Issues

ChatGPT is a Remarkable Tool -- For Experts

An empirical study of ChatGPT-3.5 on question answering and code maintenance

Extending the Frontier of ChatGPT: Code Generation and Debugging

Students' Experiences of Using ChatGPT in an Undergraduate Programming Course

Inappropriate Benefits and Identification of ChatGPT Misuse in Programming Tests: A Controlled Experiment

Analysis ChatGPT Potential: Transforming Software Development with AI Chat Bots

ChatGPT in a programming course: benefits and limitations

Fight Fire with Fire: How Much Can We Trust ChatGPT on Source Code-Related Tasks?

Exploring the Use of ChatGPT as a Tool for Learning and Assessment in Undergraduate Computer Science Curriculum: Opportunities and Challenges

An era of ChatGPT as a significant futuristic support tool: A study on features, abilities, and challenges

The advantages and limitations of using ChatGPT to enhance technological research

ChatGPT, Can You Generate Solutions for my Coding Exercises? An Evaluation on its Effectiveness in an undergraduate Java Programming Course

ChatGPT in the classroom. Exploring its potential and limitations in a Functional Programming course