Abstract:Knowledge Graphs (KGs) can serve as reliable knowledge sources for question answering (QA) due to their structured representation of knowledge. Existing research on the utilization of KG for large language models (LLMs) prevalently relies on subgraph retriever or iterative prompting, overlooking the potential synergy of LLMs' step-wise reasoning capabilities and KGs' structural nature. In this paper, we present DoG (Decoding on Graphs), a novel framework that facilitates a deep synergy between LLMs and KGs. We first define a concept, well-formed chain, which consists of a sequence of interrelated fact triplets on the KGs, starting from question entities and leading to answers. We argue that this concept can serve as a principle for making faithful and sound reasoning for KGQA. To enable LLMs to generate well-formed chains, we propose graph-aware constrained decoding, in which a constraint derived from the topology of the KG regulates the decoding process of the LLMs. This constrained decoding method ensures the generation of well-formed chains while making full use of the step-wise reasoning capabilities of LLMs. Based on the above, DoG, a training-free approach, is able to provide faithful and sound reasoning trajectories grounded on the KGs. Experiments across various KGQA tasks with different background KGs demonstrate that DoG achieves superior and robust performance. DoG also shows general applicability with various open-source LLMs.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to address the limitations of existing methods in the knowledge graph question answering (KGQA) task. Specifically, the authors point out that current methods of using large language models (LLMs) for KGQA mainly rely on sub - graph retrievers or iterative prompting, and fail to fully utilize the step - by - step reasoning ability of LLMs and the structural characteristics of knowledge graphs (KGs). To this end, the authors propose a new framework - **Decoding on Graphs (DoG)** to achieve deep synergy between LLMs and KGs. #### Main problems and solutions: 1. **Limitations of existing methods**: - **Sub - graph retriever methods**: These methods rely on specialized sub - graph retrievers to extract sub - graphs related to the question from the complete knowledge graph, and then input these sub - graphs into LLMs to predict the answer. This method makes LLMs limited to selecting the most likely answer when answering questions, without deeply participating in the graph reasoning process. - **Iterative prompting methods**: These methods let LLMs perform a series of operations (such as exploring neighboring entities, path decision - making) on the KG through iterative prompting, and finally obtain the answer. However, this method has high requirements for the capabilities of LLMs, especially for small LLMs, it is more difficult to understand and execute complex instructions. 2. **Proposed solutions**: - **Define "well - formed chain"**: This is a sequence composed of multiple inter - related fact triples, starting from the question entity and finally leading to the answer. Such a chain can ensure that the reasoning process is faithfully and reasonably based on the KG. - **Graph - aware constrained decoding**: In order to make LLMs generate "well - formed chains", the authors propose a graph - aware constrained decoding method. This method imposes constraints on the decoding process of LLMs to ensure that the generated chains conform to the topological structure of the KG and fully utilize the step - by - step reasoning ability of LLMs. - **Beam search execution**: In order to reduce the impact of error propagation, the authors also integrate beam search execution, allowing multiple valid triples to be considered in each reasoning step. Through these innovations, the DoG framework can provide faithful and reasonable reasoning trajectories without additional training, and shows superior performance in multiple KGQA benchmark tests. ### Summary The core objective of this paper is to introduce the concept of "well - formed chain" and the graph - aware constrained decoding method, so that LLMs can better use the structural information of KGs for reasoning, thereby improving the accuracy and robustness of the KGQA task.

Decoding on Graphs: Faithful and Sound Reasoning on Knowledge Graphs through Generation of Well-Formed Chains

Debate on Graph: a Flexible and Reliable Reasoning Framework for Large Language Models

Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models

Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph

Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answering

Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning

Think-on-Graph: Deep and Responsible Reasoning of Large Language Model with Knowledge Graph

Reasoning on Efficient Knowledge Paths:Knowledge Graph Guides Large Language Model for Domain Question Answering

Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation

Direct Evaluation of Chain-of-Thought in Multi-hop Reasoning with Knowledge Graphs

Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoning

ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graph

Chain-of-Knowledge: Integrating Knowledge Reasoning into Large Language Models by Learning from Knowledge Graphs

Knowledge Graph Reasoning with Logics and Embeddings: Survey and Perspective

Explore then Determine: A GNN-LLM Synergy Framework for Reasoning over Knowledge Graph

Neural Axiom Network for Knowledge Graph Reasoning

FiDeLiS: Faithful Reasoning in Large Language Model for Knowledge Graph Question Answering

Right for Right Reasons: Large Language Models for Verifiable Commonsense Knowledge Graph Question Answering

An Enhanced Prompt-Based LLM Reasoning Scheme via Knowledge Graph-Integrated Collaboration