Abstract:The task of reasoning over Knowledge Graphs (KGs) poses a significant challenge for Large Language Models (LLMs) due to the complex structure and large amounts of irrelevant information. Existing LLM reasoning methods overlook the importance of compositional learning on KG to supply with precise knowledge. Besides, the fine-tuning and frequent interaction with LLMs incur substantial time and resource costs. This paper focuses on the Question Answering over Knowledge Graph (KGQA) task and proposes an Explore-then-Determine (EtD) framework that synergizes LLMs with graph neural networks (GNNs) for reasoning over KGs. The Explore stage employs a lightweight GNN to explore promising candidates and relevant fine-grained knowledge to the questions, while the Determine stage utilizes the explored information to construct a knowledge-enhanced multiple-choice prompt, guiding a frozen LLM to determine the final answer. Extensive experiments on three benchmark KGQA datasets demonstrate that EtD achieves state-of-the-art performance and generates faithful reasoning results.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: the challenges faced by large language models (LLMs) when performing reasoning on knowledge graphs (KG). Specifically, LLMs have the following problems when handling KGQA (Question Answering over Knowledge Graph) tasks: 1. **Complex structure and a large amount of irrelevant information**: The complex structure of the knowledge graph and a large amount of irrelevant information make it difficult for LLMs to reason effectively. 2. **Lack of the importance of combinatorial learning**: Existing methods overlook the importance of performing combinatorial learning on the knowledge graph to provide accurate knowledge. 3. **High time and resource costs**: The time and resource costs caused by fine - tuning and frequent interactions are high. To solve these problems, the paper proposes a framework named "Explore - then - Determine (EtD)", which combines graph neural networks (GNNs) and large language models (LLMs) to perform reasoning on the knowledge graph more effectively. ### Main features of the EtD framework: - **Explore stage**: Use a lightweight GNN to explore promising candidate answers and fine - grained knowledge related to the question, and adaptively filter out irrelevant information. - **Determine stage**: Use the explored information to construct a knowledge - enhanced multiple - choice prompt to guide the frozen LLM to determine the final answer. In this way, the EtD framework can perform reasoning on the knowledge graph more efficiently and accurately, and reduces the need for training and fine - tuning of the LLM. ### Specific contributions of the paper: 1. Propose a novel "Explore - then - Determine" framework, which combines the precision of combinatorial learning and the powerful ability of language understanding. 2. Design a lightweight GNN model empowered by LLM for accurate and efficient KG exploration, and propose a knowledge - enhanced multiple - choice prompt to ensure the accuracy of the answer. 3. Through extensive experimental verification, this method has achieved state - of - the - art performance on three benchmark KGQA datasets, demonstrating its efficient and interpretable reasoning ability. In summary, this paper aims to solve the key challenges in existing KGQA methods by combining the advantages of GNN and LLM, thereby improving the accuracy and efficiency of reasoning.

Explore then Determine: A GNN-LLM Synergy Framework for Reasoning over Knowledge Graph

Debate on Graph: a Flexible and Reliable Reasoning Framework for Large Language Models

Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answering

ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graph

KnowledgeNavigator: Leveraging Large Language Models for Enhanced Reasoning over Knowledge Graph

An Enhanced Prompt-Based LLM Reasoning Scheme via Knowledge Graph-Integrated Collaboration

Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph

Decoding on Graphs: Faithful and Sound Reasoning on Knowledge Graphs through Generation of Well-Formed Chains

KG-GPT: A General Framework for Reasoning on Knowledge Graphs Using Large Language Models

Reasoning on Efficient Knowledge Paths:Knowledge Graph Guides Large Language Model for Domain Question Answering

Exploring Large Language Models for Knowledge Graph Completion

LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities

QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering

Think-on-Graph: Deep and Responsible Reasoning of Large Language Model with Knowledge Graph

Logic Query of Thoughts: Guiding Large Language Models to Answer Complex Logic Queries with Knowledge Graphs

Knowledge Graph-Enhanced Large Language Models via Path Selection

Exploring & Exploiting High-Order Graph Structure for Sparse Knowledge Graph Completion

LLM as Prompter: Low-resource Inductive Reasoning on Arbitrary Knowledge Graphs

CuriousLLM: Elevating Multi-Document QA with Reasoning-Infused Knowledge Graph Prompting