Meta-Tuning LLMs to Leverage Lexical Knowledge for Generalizable Language Style Understanding

Ruohao Guo,Wei Xu,Alan Ritter
2024-06-06
Abstract:Language style is often used by writers to convey their intentions, identities, and mastery of language. In this paper, we show that current large language models struggle to capture some language styles without fine-tuning. To address this challenge, we investigate whether LLMs can be meta-trained based on representative lexicons to recognize new styles they have not been fine-tuned on. Experiments on 13 established style classification tasks, as well as 63 novel tasks generated using LLMs, demonstrate that meta-training with style lexicons consistently improves zero-shot transfer across styles. We release the code and data at <a class="link-external link-http" href="http://github.com/octaviaguo/Style-LLM" rel="external noopener nofollow">this http URL</a> .
Computation and Language,Machine Learning
What problem does this paper attempt to address?
This paper mainly discusses the limitations of large language models (LLMs) in understanding and classifying language styles, especially for new styles without fine-tuning. The study proposes a vocabulary-based meta-training method to enhance LLMs' zero-shot style recognition ability by leveraging representative vocabularies. Experiments on 13 established style classification tasks and 63 new tasks generated by LLMs demonstrate that meta-training with vocabularies can significantly improve the model's generalization performance for new styles. Furthermore, the study finds that randomizing category names helps prevent the model from simply memorizing style labels of training examples, but truly learning vocabularies and understanding language styles. Finally, the paper provides benchmark datasets and code for zero-shot style classification.