Abstract:The task of predicting multiple links within knowledge graphs (KGs) stands as a challenge in the field of knowledge graph analysis, a challenge increasingly resolvable due to advancements in natural language processing (NLP) and KG embedding techniques. This paper introduces a novel methodology, the Knowledge Graph Large Language Model Framework (KG-LLM), which leverages pivotal NLP paradigms, including chain-of-thought (CoT) prompting and in-context learning (ICL), to enhance multi-hop link prediction in KGs. By converting the KG to a CoT prompt, our framework is designed to discern and learn the latent representations of entities and their interrelations. To show the efficacy of the KG-LLM Framework, we fine-tune three leading Large Language Models (LLMs) within this framework, employing both non-ICL and ICL tasks for a comprehensive evaluation. Further, we explore the framework's potential to provide LLMs with zero-shot capabilities for handling previously unseen prompts. Our experimental findings discover that integrating ICL and CoT not only augments the performance of our approach but also significantly boosts the models' generalization capacity, thereby ensuring more precise predictions in unfamiliar scenarios.

What problem does this paper attempt to address?

The paper primarily aims to address the issue of multi-hop link prediction in Knowledge Graphs (KGs) and proposes a new method—the Knowledge Graph Large Language Model Framework (KG-LLM). Below is an overview of the specific problems the paper attempts to solve: ### Research Background and Problem Definition 1. **Challenges of Multi-Hop Link Prediction**: Although there have been good results for direct link prediction between two nodes, there is still significant difficulty in handling cases that require inferring the final connection through multiple intermediate entities (i.e., multi-hop link prediction). This requires the model to understand the intermediate entities and their relationships and to perform logical reasoning. 2. **Explainability Issues**: Existing knowledge graph models lack clear explanation steps during prediction, making debugging difficult and affecting accuracy. 3. **Lack of Generalization Ability**: Traditional models perform poorly when faced with unseen tasks, lacking sufficient generalization ability. ### Proposed Method 1. **Introduction of the KG-LLM Framework**: The paper proposes a novel method—the KG-LLM framework, which leverages natural language processing techniques to convert structured knowledge graph data into natural language prompts and uses these prompts to fine-tune large language models (LLMs) to enhance their multi-hop link prediction capabilities on knowledge graphs. 2. **Chain-of-Thought (CoT)**: The framework converts knowledge graph paths into chain-of-thought prompts, enabling the model to better understand and learn the latent representations of entities and their relationships. 3. **Instruction Fine-Tuning (IFT)**: The LLMs are trained using IFT techniques to perform tasks based on the provided natural language prompts. 4. **In-Context Learning (ICL)**: By incorporating ICL techniques, the model not only improves performance but also gains the ability to handle previously unseen prompts, demonstrating the method's innovation and ability to address multi-hop link prediction challenges. ### Main Contributions 1. **Chain-of-Thought Prompts**: By converting knowledge graphs into chain-of-thought prompts, LLMs can better understand entities and their relationships. 2. **Validation on Real-World Datasets**: Analysis of real-world datasets demonstrates that the framework significantly enhances the generative performance of multi-hop link prediction in knowledge graphs. 3. **Enhanced Generalization Ability**: Experimental results show that the framework significantly improves the LLMs' ability to respond to unseen prompts. ### Overview of the Experimental Section - The paper uses two real-world datasets, WN18RR and NELL-995, for experiments and provides detailed experimental setup descriptions. - Without using in-context learning, the performance of the KG-LLM framework significantly outperforms traditional methods and methods using only chain-of-thought prompts. - When in-context learning is introduced, the model's performance under the KG-LLM framework further improves, especially in multi-hop link prediction tasks. - In multi-hop relationship prediction tasks, the KG-LLM framework also shows better results than methods using only chain-of-thought prompts, particularly when combined with in-context learning.

Knowledge Graph Large Language Model (KG-LLM) for Link Prediction

Knowledge Graph-Enhanced Large Language Models via Path Selection

LPNL: Scalable Link Prediction with Large Language Models

Large Language Models Can Better Understand Knowledge Graphs Than We Thought

Retrieval-Augmented Language Model for Extreme Multi-Label Knowledge Graph Link Prediction

LLM as Prompter: Low-resource Inductive Reasoning on Arbitrary Knowledge Graphs

Large Language Models and Knowledge Graphs: Opportunities and Challenges

Exploring Large Language Models for Knowledge Graph Completion

LinkGPT: Teaching Large Language Models To Predict Missing Links

LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities

Comprehending Knowledge Graphs with Large Language Models for Recommender Systems

Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completion

Combining Knowledge Graphs and Large Language Models

A Condensed Transition Graph Framework for Zero-shot Link Prediction with Large Language Models

Chain-of-Knowledge: Integrating Knowledge Reasoning into Large Language Models by Learning from Knowledge Graphs

Reasoning on Efficient Knowledge Paths:Knowledge Graph Guides Large Language Model for Domain Question Answering

Graph Neural Prompting with Large Language Models

Clue-Guided Path Exploration: Optimizing Knowledge Graph Retrieval with Large Language Models to Address the Information Black Box Challenge

Cross-KG Link Prediction by Learning Substructural Semantics

Retrieval and Reasoning on KGs: Integrate Knowledge Graphs into Large Language Models for Complex Question Answering

Synergizing Knowledge Graphs with Large Language Models: A Comprehensive Review and Future Prospects