Abstract:Recent advancements in automatic code generation using large language model (LLM) agent have brought us closer to the future of automated software development. However, existing single-agent approaches face limitations in generating and improving large-scale, complex codebases due to constraints in context length. To tackle this challenge, we propose Self-Organized multi-Agent framework (SoA), a novel multi-agent framework that enables the scalable and efficient generation and optimization of large-scale code. In SoA, self-organized agents operate independently to generate and modify code components while seamlessly collaborating to construct the overall codebase. A key feature of our framework is the automatic multiplication of agents based on problem complexity, allowing for dynamic scalability. This enables the overall code volume to be increased indefinitely according to the number of agents, while the amount of code managed by each agent remains constant. We evaluate SoA on the HumanEval benchmark and demonstrate that, compared to a single-agent system, each agent in SoA handles significantly less code, yet the overall generated code is substantially greater. Moreover, SoA surpasses the powerful single-agent baseline by 5% in terms of Pass@1 accuracy.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the problem of scalability limitations faced by a single - agent in generating and optimizing large - scale complex code. Specifically, the existing single - agent methods based on large language models (LLM) encounter the following challenges when dealing with complex implementations that require large codebases: 1. **Context length limitation**: A single agent has to manage the entire code - generation process. As the codebase grows, the requirement for context length increases, leading to a decline in inference performance. 2. **Code management difficulties**: When the codebase becomes very large, it is difficult for a single agent to effectively generate, modify, and manage the code. 3. **Self - debugging pressure**: A complex codebase makes it more difficult to understand and manage the context during the self - debugging process. To solve these problems, the paper proposes a new multi - agent framework - **Self - Organized Agents (SoA)**. The key features of SoA include: - **Self - organizing ability**: Agents proliferate automatically according to the complexity of the problem, enabling the overall codebase to be infinitely expandable while the amount of code managed by each agent remains constant. - **Distributed collaboration**: Multiple agents independently generate and modify code components while seamlessly collaborating to build the entire codebase. - **Dynamic scalability**: The number of agents can be adjusted dynamically according to the complexity of the problem, ensuring efficient generation and optimization of large - scale code. Through these features, the SoA framework can overcome the limitations of single - agent methods in handling large - scale complex code and provide a more flexible and efficient solution. ### Experimental results The experimental results show that SoA outperforms the powerful single - agent baseline Reflexion in the HumanEval benchmark test, with a 5% improvement in Pass@1 accuracy. In addition, SoA can significantly increase the total amount of generated code without increasing the burden on each agent, demonstrating its excellent scalability and efficiency. ### Summary By introducing the SoA framework, this paper solves the scalability problem of a single - agent in generating and optimizing large - scale complex code, providing new ideas and technical means for automated software development.

Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimization

EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms

CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges

MegaAgent: A Practical Framework for Autonomous Cooperation in Large-Scale LLM Agent Systems

AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation

CodePori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology

A Multi-AI Agent System for Autonomous Optimization of Agentic AI Solutions via Iterative Refinement and LLM-Driven Feedback Loops

SMoA: Improving Multi-agent Large Language Models with Sparse Mixture-of-Agents

ProgAI: Enhancing Code Generation with LLMs For Real World Challenges

Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents

MOSS: Enabling Code-Driven Evolution and Context Management for AI Agents

LLM Agent Operating System

Very Large-Scale Multi-Agent Simulation in AgentScope

Agentless: Demystifying LLM-based Software Engineering Agents

MASAI: Modular Architecture for Software-engineering AI Agents

AgentSquare: Automatic LLM Agent Search in Modular Design Space

Autonomous Multi-Objective Optimization Using Large Language Model

LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead

Synergistic Simulations: Multi-Agent Problem Solving with Large Language Models

Code Generation Using Self-Interactive Assistant