Abstract:Extensive knowledge graphs (KGs) have been constructed to facilitate knowledge-driven tasks across various scenarios. However, existing work usually develops separate reasoning models for different KGs, lacking the ability to generalize and transfer knowledge across diverse KGs and reasoning settings. In this paper, we propose a prompt-based KG foundation model via in-context learning, namely KG-ICL, to achieve a universal reasoning ability. Specifically, we introduce a prompt graph centered with a query-related example fact as context to understand the query relation. To encode prompt graphs with the generalization ability to unseen entities and relations in queries, we first propose a unified tokenizer that maps entities and relations in prompt graphs to predefined tokens. Then, we propose two message passing neural networks to perform prompt encoding and KG reasoning, respectively. We conduct evaluation on 43 different KGs in both transductive and inductive settings. Results indicate that the proposed KG-ICL outperforms baselines on most datasets, showcasing its outstanding generalization and universal reasoning capabilities. The source code is accessible on GitHub: <a class="link-external link-https" href="https://github.com/nju-websoft/KG-ICL" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The core problem that this paper attempts to solve is the insufficient generalization and transfer ability of existing knowledge graph (KGs) reasoning models between different KGs. Specifically: 1. **Limitations of Existing Work**: - Existing work usually develops independent reasoning models for different KGs and lacks the knowledge transfer ability across multiple KGs and reasoning scenarios. - Many methods can only handle reasoning tasks in static KGs and are unable to deal with newly emerging entities or relations (i.e., the inductive reasoning problem). - Even if some methods can handle new entities, they are still difficult to perform reasoning on unseen KGs because the entity and relation vocabularies between these KGs are not shared and not linked. 2. **The Method Proposed in This Paper**: - To overcome the above challenges, the author proposes a prompt - based knowledge graph foundation model - **KG - ICL**, which achieves general reasoning ability through in - context learning. - The main innovations of **KG - ICL** include: - **Prompt Graph**: Construct a prompt graph around query - related example facts to understand the query relation and capture the local context related to the query. - **Unified Tokenizer**: Map entities and relations in different KGs to predefined tokens to ensure that the model can be generalized to unseen entities and relations. - **Message Passing Neural Networks (MPNNs)**: Used to encode the prompt graph and perform KG reasoning, improving the generalization ability and reasoning performance of the model. 3. **Experimental Verification**: - The author evaluated on 43 different KG datasets, covering inductive, fully - inductive, and transductive settings. - The experimental results show that **KG - ICL** outperforms the baseline model on most datasets, demonstrating its excellent generalization and general reasoning abilities. In summary, this paper aims to solve the problem of insufficient generalization and transfer ability of existing methods between different KGs by introducing a new prompt - based KG reasoning model, thereby achieving more extensive reasoning abilities.

A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoning

Structure Pre-training and Prompt Tuning for Knowledge Graph Transfer

Prompt-fused framework for Inductive Logical Query Answering

Neural Axiom Network for Knowledge Graph Reasoning

Towards Foundation Models for Knowledge Graph Reasoning

Schema-aware Reference As Prompt Improves Data-Efficient Knowledge Graph Construction

LLM as Prompter: Low-resource Inductive Reasoning on Arbitrary Knowledge Graphs

Knowledge Graph Prompting for Multi-Document Question Answering

Graph Neural Prompting with Large Language Models

Towards Graph Foundation Models: The Perspective of Zero-shot Reasoning on Knowledge Graphs

Retrieval, Reasoning, Re-ranking: A Context-Enriched Framework for Knowledge Graph Completion

Simple knowledge graph completion model based on PU learning and prompt learning

One Subgraph for All: Efficient Reasoning on Opening Subgraphs for Inductive Knowledge Graph Completion

Retrieval and Reasoning on KGs: Integrate Knowledge Graphs into Large Language Models for Complex Question Answering

Rule-enhanced Iterative Complementation for Knowledge Graph Reasoning

Contextual Knowledge Graph

An Enhanced Prompt-Based LLM Reasoning Scheme via Knowledge Graph-Integrated Collaboration

UniKGQA: Unified Retrieval and Reasoning for Solving Multi-hop Question Answering Over Knowledge Graph

Information bottleneck based knowledge selection for commonsense reasoning

G-SAP: Graph-based Structure-Aware Prompt Learning over Heterogeneous Knowledge for Commonsense Reasoning