Abstract:In this paper, we explore the capabilities of LLMs in capturing lexical-semantic knowledge from WordNet on the example of the LLaMA-2-7b model and test it on multiple lexical semantic tasks. As the outcome of our experiments, we present TaxoLLaMA, the everything-in-one model, lightweight due to 4-bit quantization and LoRA. It achieves 11 SotA results, 4 top-2 results out of 16 tasks for the Taxonomy Enrichment, Hypernym Discovery, Taxonomy Construction, and Lexical Entailment tasks. Moreover, it demonstrates very strong zero-shot performance on Lexical Entailment and Taxonomy Construction with no fine-tuning. We also explore its hidden multilingual and domain adaptation capabilities with a little tuning or few-shot learning. All datasets, code, and model are available online at <a class="link-external link-https" href="https://github.com/VityaVitalich/TaxoLLaMA" rel="external noopener nofollow">this https URL</a>

What problem does this paper attempt to address?

### Problems Addressed by the Paper This paper explores the ability of large language models (LLMs) to capture lexical semantic knowledge in WordNet, using the LLaMA-2-7b model as an example, and tests it on multiple lexical semantic tasks. Specifically, this paper proposes the TaxoLLaMA model, a lightweight "one-stop" model for handling classification-related tasks such as taxonomy enrichment, hypernym discovery, taxonomy construction, and lexical entailment. ### Main Issues and Goals 1. **Hypernym Discovery**: - Task Description: Given a hyponym, predict its list of hypernyms. - Problem Background: Existing research has limited applications of LLMs in hypernym discovery tasks across different domains and languages, especially in verifying performance in multilingual and specific domains. 2. **Taxonomy Enrichment**: - Task Description: Add new words to the correct hypernym (node) in a given taxonomy. - Problem Background: Existing methods typically rely on complex pipelines, whereas this paper aims to directly generate correct hypernym relationships through LLMs. 3. **Lexical Entailment**: - Task Description: Identify the semantic relationship between phrase pairs, such as the hyponym "cat" entailing the hypernym "animal." - Problem Background: Existing models need improvement in zero-shot settings, particularly in adaptability across different languages and domains. 4. **Taxonomy Construction**: - Task Description: Extract hypernym-hyponym relationships from a given list of domain-specific terms and construct a domain taxonomy. - Problem Background: Existing models often use complex graph neural network (GNN) architectures, while this paper aims to simplify this process through LLMs. ### Solutions - **Model Design**: TaxoLLaMA is based on the LLaMA-2-7b model, optimized using 4-bit quantization and LoRA technology to run more efficiently on GPU devices. - **Dataset Construction**: Sample hypernym-hyponym relationships of nouns and verbs from WordNet-3.0 to build an instruction-tuning dataset. - **Training Methods**: Use both generative and ranking methods to adapt to different task requirements. - **Experimental Results**: TaxoLLaMA achieved SOTA (State-of-the-Art) or near-SOTA results on multiple tasks, particularly excelling in zero-shot settings. ### Contributions 1. **Introduced a method using LLMs to solve various lexical semantic tasks**, proposing a suitable taxonomy instruction-tuning method through hypernym prediction. 2. **Released the TaxoLLaMA model**, which achieved SOTA results on 11 out of 16 tasks and ranked second on 4 tasks. 3. **Provided an instruction-tuning dataset based on English WordNet-3.0**, and collected input word definitions from taxonomy enrichment and lexical entailment datasets. 4. **Conducted detailed error analysis**, using manual and automatic methods to evaluate error patterns and model quality across all tasks. Through these contributions, this paper not only validates the potential of LLMs in lexical semantic tasks but also provides valuable resources and methods for future research.

TaxoLLaMA: WordNet-based Model for Solving Multiple Lexical Semantic Tasks

LLMs4OL: Large Language Models for Ontology Learning

End-to-End Ontology Learning with Large Language Models

PolyLM: An Open Source Polyglot Large Language Model

QuickLLaMA: Query-aware Inference Acceleration for Large Language Models

LLMaAA: Making Large Language Models as Active Annotators

PMC-LLaMA: Further Finetuning LLaMA on Medical Papers

LLaVaOLMoBitnet1B: Ternary LLM goes Multimodal!

TeacherLM: Teaching to Fish Rather Than Giving the Fish, Language Modeling Likewise

LexMatcher: Dictionary-centric Data Collection for LLM-based Machine Translation

MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models

LlamaLens: Specialized Multilingual LLM for Analyzing News and Social Media Content

SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models

Large Language Models aren't all that you need

Large Language Models for Scholarly Ontology Generation: An Extensive Analysis in the Engineering Field

Open Llama2 Model for the Lithuanian Language

VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs

PLLaMa: An Open-source Large Language Model for Plant Science

MaLA-500: Massive Language Adaptation of Large Language Models

How Vocabulary Sharing Facilitates Multilingualism in LLaMA?

PMC-LLaMA: Towards Building Open-source Language Models for Medicine