A prompt construction method for the reverse dictionary task of large-scale language models
Sicheng Tian,Shaobin Huang,Rongsheng Li,Chi Wei
DOI: https://doi.org/10.1016/j.engappai.2024.108596
IF: 8
2024-05-17
Engineering Applications of Artificial Intelligence
Abstract:A reverse dictionary (RD) is a tool designed to find the appropriate word based on a given definition. While large language models (LLMs) have demonstrated exceptional natural language generation capabilities, offering a promising solution for this task, existing LLMs, particularly those with smaller parameter sizes, still require performance enhancements on RD. Fine-tuning LLMs is a direct method of improvement, but it comes with high computational costs and the potential sacrifice of the LLMs' ability to solve other problems. In contrast, employing prompt engineering to construct high-quality prompts that evoke the latent potential of LLMs on RD is a more ideal approach. To this end, we propose a method of automatically constructing prompts based on a small-scale language model. Our method analyzes definitions across multiple semantic dimensions and introduces negative samples to enhance the model's universality and adaptability. Consequently, our small model can generate a candidate set containing the target word with an accuracy rate of 93.1% . We then use these candidates to form effective prompts for LLMs, significantly improving the performance of larger models. Evaluations on a new dataset from the Oxford dictionary show that our LLMs can directly select the target word with an accuracy of 57.5% , which increases to 73.8% when given five candidate words. Our research not only enhances the performance of LLMs on RD but also provides a new perspective on how small models can augment large models, which is particularly important when computational resources are limited.
automation & control systems,computer science, artificial intelligence,engineering, electrical & electronic, multidisciplinary