Abstract:Large language models (LLMs) have not only revolutionized natural language processing but also extended their prowess to various domains, marking a significant stride towards artificial general intelligence. The interplay between LLMs and evolutionary algorithms (EAs), despite differing in objectives and methodologies, share a common pursuit of applicability in complex problems. Meanwhile, EA can provide an optimization framework for LLM's further enhancement under black-box settings, empowering LLM with flexible global search capacities. On the other hand, the abundant domain knowledge inherent in LLMs could enable EA to conduct more intelligent searches. Furthermore, the text processing and generative capabilities of LLMs would aid in deploying EAs across a wide range of tasks. Based on these complementary advantages, this paper provides a thorough review and a forward-looking roadmap, categorizing the reciprocal inspiration into two main avenues: LLM-enhanced EA and EA-enhanced LLM. Some integrated synergy methods are further introduced to exemplify the complementarity between LLMs and EAs in diverse scenarios, including code generation, software engineering, neural architecture search, and various generation tasks. As the first comprehensive review focused on the EA research in the era of LLMs, this paper provides a foundational stepping stone for understanding the collaborative potential of LLMs and EAs. The identified challenges and future directions offer guidance for researchers and practitioners to unlock the full potential of this innovative collaboration in propelling advancements in optimization and artificial intelligence. We have created a GitHub repository to index the relevant papers: <a class="link-external link-https" href="https://github.com/wuxingyu-ai/LLM4EC" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### Problems the Paper Aims to Address This paper aims to explore the dynamic interaction and synergistic effects between Evolutionary Algorithms (EAs) and Large Language Models (LLMs) in the era of large-scale language models. Specifically, the goals of the paper include: 1. **Exploring Complementary Advantages**: Although LLMs and EAs have significant differences in their goals and methods, they both aim to be applied to complex problems. LLMs achieve cross-task unified processing by learning from large amounts of data, while EAs, as general solvers, have low data dependency when solving a wide range of problems with different characteristics. Therefore, both exhibit complementary advantages in solving complex problems with vast search spaces and uncertain environments. 2. **Addressing the Limitations of LLMs**: Due to the black-box nature and limited search capabilities of LLMs, EAs can provide an optimization framework to further enhance the performance of LLMs. For example, EAs can iteratively optimize solutions generated by LLMs to improve the quality of results; additionally, EAs can achieve more flexible searches by designing well-structured search spaces and evolutionary operators. 3. **Leveraging the Advantages of LLMs**: LLMs excel in general knowledge, text understanding, and generation capabilities, which can compensate for certain limitations encountered when EAs are applied independently. The rich prior knowledge of LLMs can provide effective guidance in the early stages of the search, accelerating the convergence of EAs to high-quality solutions; their powerful text generation and processing capabilities also make EAs more convenient in tasks involving text generation. 4. **Proposing Integrated Methods**: The paper introduces several integrated collaborative methods, demonstrating the complementarity of LLMs and EAs in various application scenarios such as code generation, software engineering, and neural architecture search. 5. **Outlook on Future Directions**: Through a critical analysis of existing research, the paper identifies several key challenges in the cross-research field of LLMs and EAs and provides guidance for future exploration. In summary, this paper aims to provide a foundation for understanding the collaborative potential of LLMs and EAs and to guide advancements in the fields of optimization and artificial intelligence.

Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap

Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap

When large language models meet evolutionary algorithms

Exploring the Improvement of Evolutionary Computation via Large Language Models

Deep Insights into Automated Optimization with Large Language Models and Evolutionary Algorithms

Large Language Models as Evolutionary Optimizers

Large Language Models as Surrogate Models in Evolutionary Algorithms: A Preliminary Study

Large Language Models for Generative Information Extraction: A Survey

A Survey on Large Language Models for Code Generation

Towards an Understanding of Large Language Models in Software Engineering Tasks

A Survey on Self-Evolution of Large Language Models

Large Language Model-Based Agents for Software Engineering: A Survey

The Rise and Potential of Large Language Model Based Agents: A Survey

Large Language Models As Evolution Strategies

Autonomous Multi-Objective Optimization Using Large Language Model

A Systematic Survey on Large Language Models for Algorithm Design

Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods

A Survey on Evaluation of Large Language Models

From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future