Prompt Optimization in Large Language Models

Antonio Sabbatella,Andrea Ponti,Ilaria Giordani,Antonio Candelieri,Francesco Archetti
DOI: https://doi.org/10.3390/math12060929
IF: 2.4
2024-03-22
Mathematics
Abstract:Prompt optimization is a crucial task for improving the performance of large language models for downstream tasks. In this paper, a prompt is a sequence of n-grams selected from a vocabulary. Consequently, the aim is to select the optimal prompt concerning a certain performance metric. Prompt optimization can be considered as a combinatorial optimization problem, with the number of possible prompts (i.e., the combinatorial search space) given by the size of the vocabulary (i.e., all the possible n-grams) raised to the power of the length of the prompt. Exhaustive search is impractical; thus, an efficient search strategy is needed. We propose a Bayesian Optimization method performed over a continuous relaxation of the combinatorial search space. Bayesian Optimization is the dominant approach in black-box optimization for its sample efficiency, along with its modular structure and versatility. We use BoTorch, a library for Bayesian Optimization research built on top of PyTorch. Specifically, we focus on Hard Prompt Tuning, which directly searches for an optimal prompt to be added to the text input without requiring access to the Large Language Model, using it as a black-box (such as for GPT-4 which is available as a Model as a Service). Albeit preliminary and based on "vanilla" Bayesian Optimization algorithms, our experiments with RoBERTa as a large language model, on six benchmark datasets, show good performances when compared against other state-of-the-art black-box prompt optimization methods and enable an analysis of the trade-off between the size of the search space, accuracy, and wall-clock time.
mathematics
What problem does this paper attempt to address?
### The Problem Addressed by This Paper This paper primarily focuses on the issue of prompt optimization in large language models (LLMs). Specifically, its goal is to enhance the model's performance on specific tasks by optimizing the prompt sequence preceding the input text without modifying the model parameters. The main contributions of the paper are as follows: 1. **Importance of Prompt Optimization**: - Prompt engineering is crucial for improving the performance of large language models on specific tasks. Manually designing effective prompts is a time-consuming and costly process. - Automated prompt optimization methods are particularly suitable for black-box scenarios, where only the model's output is accessible, not the gradients. 2. **Research Focus**: - The paper focuses on Hard Prompt Tuning (HPT), which involves directly searching for the optimal prompt sequence in the combinatorial search space. - A Bayesian optimization-based method is proposed, which relaxes the combinatorial search space into a continuous space to improve search efficiency. 3. **Methodology**: - Bayesian Optimization (BO) is employed to search for the optimal prompt. Through continuous relaxation techniques, the originally discrete prompt space is converted into a continuous space, thereby leveraging the efficient sampling characteristics of BO. - The BoTorch library is used to implement the Bayesian optimization algorithm, and the effectiveness of this method is validated on multiple benchmark datasets. 4. **Experimental Results**: - Experiments were conducted on six standard datasets, including MNLI, QQP, SST-2, etc. The results show that the proposed HPT method outperforms other black-box prompt optimization methods in terms of accuracy and runtime. In summary, this paper aims to enhance the performance of large language models on specific tasks by proposing an efficient prompt optimization method, especially in black-box environments.