Large Language Models As Evolution Strategies

Robert Tjarko Lange,Yingtao Tian,Yujin Tang

2024-02-28

Abstract:Large Transformer models are capable of implementing a plethora of so-called in-context learning algorithms. These include gradient descent, classification, sequence completion, transformation, and improvement. In this work, we investigate whether large language models (LLMs), which never explicitly encountered the task of black-box optimization, are in principle capable of implementing evolutionary optimization algorithms. While previous works have solely focused on language-based task specification, we move forward and focus on the zero-shot application of LLMs to black-box optimization. We introduce a novel prompting strategy, consisting of least-to-most sorting of discretized population members and querying the LLM to propose an improvement to the mean statistic, i.e. perform a type of black-box recombination operation. Empirically, we find that our setup allows the user to obtain an LLM-based evolution strategy, which we call `EvoLLM', that robustly outperforms baseline algorithms such as random search and Gaussian Hill Climbing on synthetic BBOB functions as well as small neuroevolution tasks. Hence, LLMs can act as `plug-in' in-context recombination operators. We provide several comparative studies of the LLM's model size, prompt strategy, and context construction. Finally, we show that one can flexibly improve EvoLLM's performance by providing teacher algorithm information via instruction fine-tuning on previously collected teacher optimization trajectories.

Artificial Intelligence,Machine Learning,Neural and Evolutionary Computing

What problem does this paper attempt to address?

The problem discussed in this paper is whether large language models (LLMs) can implement evolutionary optimization algorithms even if they have never directly encountered black-box optimization tasks. In the study, the authors propose a new hint strategy that uses sorting and querying LLMs to propose improved average statistics through a black-box recombination operation. The experiments show that this LLM-based evolutionary strategy (EvoLLM) outperforms traditional baseline algorithms such as random search and Gaussian hill climbing on synthetic black-box optimization functions and small-scale neural evolution tasks. In addition, the paper investigates the impact of LLM model size, hint strategy, and context construction on performance, and demonstrates that fine-tuning EvoLLM through teacher algorithm information can further improve its performance. Overall, the paper aims to explore the ability of LLMs as general pattern machines and apply them to black-box optimization without the need for pre-training.