Abstract:This paper introduces AutoSurvey, a speedy and well-organized methodology for automating the creation of comprehensive literature surveys in rapidly evolving fields like artificial intelligence. Traditional survey paper creation faces challenges due to the vast volume and complexity of information, prompting the need for efficient survey methods. While large language models (LLMs) offer promise in automating this process, challenges such as context window limitations, parametric knowledge constraints, and the lack of evaluation benchmarks remain. AutoSurvey addresses these challenges through a systematic approach that involves initial retrieval and outline generation, subsection drafting by specialized LLMs, integration and refinement, and rigorous evaluation and iteration. Our contributions include a comprehensive solution to the survey problem, a reliable evaluation method, and experimental validation demonstrating AutoSurvey's effectiveness.We open our resources at \url{<a class="link-external link-https" href="https://github.com/AutoSurveys/AutoSurvey" rel="external noopener nofollow">this https URL</a>}.

What problem does this paper attempt to address?

The paper attempts to address the challenges faced in writing traditional survey papers in rapidly evolving fields such as artificial intelligence. Specifically, these challenges include: 1. **Large and Complex Information**: With the rapid development of research, the number of new papers is increasing quickly, making it increasingly difficult to write survey papers. For example, in the first 4 months of 2024 alone, more than 4,000 papers containing the keyword "Large Language Model (LLM)" were submitted to arXiv. 2. **Inefficiency of Existing Methods**: Traditional methods of writing survey papers rely on manual efforts, requiring a significant amount of time and energy to read, organize, and synthesize a large volume of research literature. This is not only time-consuming but also challenging to keep up with the rapid updates in new research. 3. **Challenges in Applying Large Language Models (LLM)**: - **Context Window Limitation**: LLMs are limited by the size of the context window when generating long texts, making it difficult to directly generate a complete survey paper. - **Parameter Knowledge Constraints**: The knowledge within LLMs may be inaccurate or outdated, making it difficult to generate high-quality citations and content. - **Lack of Evaluation Benchmarks**: There is currently a lack of reliable evaluation metrics to measure the quality of survey papers generated by LLMs. Relying on manual evaluation is both time-consuming and lacks scalability. To address these issues, the paper proposes AutoSurvey, a fast and systematic method that utilizes multiple LLMs to automatically write comprehensive survey papers. The main innovations of AutoSurvey include: - **Logical Parallel Generation**: Through a multi-stage generation method, it first generates a detailed outline, then generates content for each part in parallel, and finally integrates and refines the content, significantly improving the generation speed and content quality. - **Real-time Knowledge Update**: Using Retrieval-Augmented Generation (RAG) technology, it ensures that the generated content reflects the latest research findings. - **Multi-LLM Evaluation**: It employs multiple LLMs as evaluation tools to generate initial evaluation metrics, which are then calibrated by human experts to ensure the accuracy and academic standards of the evaluation. Through these innovations, AutoSurvey aims to improve the efficiency and quality of writing survey papers, making them better suited to rapidly evolving research fields.

AutoSurvey: Large Language Models Can Automatically Write Surveys