Abstract:Large Language Models (LLMs) may suffer from hallucinations in real-world applications due to the lack of relevant knowledge. In contrast, knowledge graphs encompass extensive, multi-relational structures that store a vast array of symbolic facts. Consequently, integrating LLMs with knowledge graphs has been extensively explored, with Knowledge Graph Question Answering (KGQA) serving as a critical touchstone for the integration. This task requires LLMs to answer natural language questions by retrieving relevant triples from knowledge graphs. However, existing methods face two significant challenges: \textit{excessively long reasoning paths distracting from the answer generation}, and \textit{false-positive relations hindering the path refinement}. In this paper, we propose an iterative interactive KGQA framework that leverages the interactive learning capabilities of LLMs to perform reasoning and Debating over Graphs (DoG). Specifically, DoG employs a subgraph-focusing mechanism, allowing LLMs to perform answer trying after each reasoning step, thereby mitigating the impact of lengthy reasoning paths. On the other hand, DoG utilizes a multi-role debate team to gradually simplify complex questions, reducing the influence of false-positive relations. This debate mechanism ensures the reliability of the reasoning process. Experimental results on five public datasets demonstrate the effectiveness and superiority of our architecture. Notably, DoG outperforms the state-of-the-art method ToG by 23.7\% and 9.1\% in accuracy on WebQuestions and GrailQA, respectively. Furthermore, the integration experiments with various LLMs on the mentioned datasets highlight the flexibility of DoG. Code is available at \url{<a class="link-external link-https" href="https://github.com/reml-group/DoG" rel="external noopener nofollow">this https URL</a>}.

Think-on-Graph: Deep and Responsible Reasoning of Large Language Model with Knowledge Graph

Think-on-Graph: Deep and Responsible Reasoning of Large Language Model with Knowledge Graph

Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph

Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation

Debate on Graph: a Flexible and Reliable Reasoning Framework for Large Language Models

Reasoning on Efficient Knowledge Paths:Knowledge Graph Guides Large Language Model for Domain Question Answering

Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models

Logic Query of Thoughts: Guiding Large Language Models to Answer Complex Logic Queries with Knowledge Graphs

Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoning

Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning

Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs

GraphLLM: Boosting Graph Reasoning Ability of Large Language Model

Chain-of-Knowledge: Integrating Knowledge Reasoning into Large Language Models by Learning from Knowledge Graphs

Decoding on Graphs: Faithful and Sound Reasoning on Knowledge Graphs through Generation of Well-Formed Chains

On Exploring the Reasoning Capability of Large Language Models with Knowledge Graphs

KnowledgeNavigator: Leveraging Large Language Models for Enhanced Reasoning over Knowledge Graph

GIVE: Structured Reasoning with Knowledge Graph Inspired Veracity Extrapolation

Plan-on-Graph: Self-Correcting Adaptive Planning of Large Language Model on Knowledge Graphs

Direct Evaluation of Chain-of-Thought in Multi-hop Reasoning with Knowledge Graphs

Graph of Thoughts: Solving Elaborate Problems with Large Language Models