Chen Qian,Wei Liu,Hongzhang Liu,Nuo Chen,Yufan Dang,Jiahao Li,Cheng Yang,Weize Chen,Yusheng Su,Xin Cong,Juyuan Xu,Dahai Li,Zhiyuan Liu,Maosong Sun
Abstract:Software development is a complex task that necessitates cooperation among multiple members with diverse skills. Numerous studies used deep learning to improve specific phases in a waterfall model, such as design, coding, and testing. However, the deep learning model in each phase requires unique designs, leading to technical inconsistencies across various phases, which results in a fragmented and ineffective development process. In this paper, we introduce ChatDev, a chat-powered software development framework in which specialized agents driven by large language models (LLMs) are guided in what to communicate (via chat chain) and how to communicate (via communicative dehallucination). These agents actively contribute to the design, coding, and testing phases through unified language-based communication, with solutions derived from their multi-turn dialogues. We found their utilization of natural language is advantageous for system design, and communicating in programming language proves helpful in debugging. This paradigm demonstrates how linguistic communication facilitates multi-agent collaboration, establishing language as a unifying bridge for autonomous task-solving among LLM agents. The code and data are available at <a class="link-external link-https" href="https://github.com/OpenBMB/ChatDev" rel="external noopener nofollow">this https URL</a>.
What problem does this paper attempt to address?
### Problems the Paper Attempts to Solve
This paper aims to address the issues of technical inconsistency and inefficiency in the software development process. Specifically:
1. **Technical Inconsistency**: Existing deep learning methods require unique designs at different stages of software development (such as design, coding, and testing), leading to technical inconsistencies between stages, making the entire development process fragmented and inefficient.
2. **Inefficient Development Process**: The traditional waterfall model divides the software development process into multiple stages, but the methods and technical designs of each stage are different, resulting in the need for independent designs at each stage from data collection to model training and inference, further reducing development efficiency.
To solve these problems, the paper proposes **ChatDev**, a chat-based software development framework that enables collaboration among multiple agents driven by large language models (LLMs) through multi-round dialogues. These agents cooperate through unified language communication during the design, coding, and testing stages to generate comprehensive solutions. The main features of ChatDev include:
- **Multi-round Dialogues**: Through multi-round dialogues, agents can gradually propose and refine solutions.
- **Communication De-illusion Mechanism**: To avoid code illusions (i.e., generated code that is incomplete, non-executable, or inaccurate), ChatDev introduces a communication de-illusion mechanism, ensuring that agents actively request more detailed information before providing formal responses.
- **Role Assignment**: Each agent has a specific role (such as requirements analyst, professional programmer, and test engineer), and role assignment is achieved through role-customized operations, improving the quality of task completion.
### Main Contributions
1. **Unified Communication Bridge**: ChatDev establishes a collaborative bridge among multiple agents through language communication, enabling smooth linking of sub-tasks in natural language and programming language, facilitating problem-solving.
2. **Reduction of Code Illusions**: The introduced communication de-illusion mechanism effectively reduces the occurrence of code illusions, improving the completeness and executability of the generated code.
3. **High-quality Software Generation**: Experimental results show that ChatDev significantly outperforms baseline methods on multiple evaluation metrics, particularly excelling in software completeness, executability, and consistency.
In summary, ChatDev provides a more efficient and consistent software development framework through multi-agent collaboration and language communication.