Abstract:Retrieval-augmented generation (RAG) has enhanced large language models (LLMs) by using knowledge retrieval to address knowledge gaps. However, existing RAG approaches often fail to ensure the depth and completeness of the information retrieved, which is essential for complex reasoning tasks. In this work, we present Think-on-Graph 2.0 (ToG-2), a hybrid RAG framework that iteratively retrieves information from both unstructured and structured knowledge sources in a tightly integrated manner. Specifically, ToG-2 leverages knowledge graphs (KGs) to connect documents via entities, facilitating deep and knowledge-guided context retrieval. Simultaneously, it uses documents as entity contexts to enable precise and efficient graph retrieval. ToG-2 alternates between graph retrieval and context retrieval to search for in-depth clues relevant to the question, enabling LLMs to generate accurate answers. We conduct a series of experiments to demonstrate the following advantages of ToG-2: (1) ToG-2 tightly integrates context retrieval and graph retrieval, enhancing context retrieval through the KG while enabling reliable graph retrieval based on contexts; (2) it achieves deep and faithful reasoning in LLMs through an iterative knowledge retrieval process that integrates contexts and the KG; and (3) ToG-2 is training-free and compatible with various LLMs as a plug-and-play solution. Extensive experiments show that ToG-2 achieves state-of-the-art (SOTA) performance on 6 out of 7 knowledge-intensive datasets with GPT-3.5, and can elevate the performance of smaller models (e.g., LLAMA-2-13B) to the level of GPT-3.5's direct reasoning.

What problem does this paper attempt to address?

### The Problem the Paper Aims to Solve This paper aims to address the issue that existing Retrieval-Augmented Generation (RAG) methods face in ensuring the depth and completeness of retrieved information when handling complex reasoning tasks. Specifically, while current RAG methods overcome the knowledge limitations of large language models (LLMs) through knowledge retrieval, they perform poorly in multi-step reasoning or tracking logical connections between different pieces of information. This is mainly because text-based RAG methods struggle to capture the structural relationships between texts, and while knowledge graphs (KGs) can effectively organize structural relationships between entities, they suffer from incompleteness and lack information beyond their ontology. ### Solution To address this, the paper proposes Think-on-Graph 2.0 (ToG-2), a hybrid RAG framework that iteratively retrieves information from both unstructured and structured knowledge sources in a tightly coupled manner. The main features of ToG-2 include: 1. **Deep Retrieval**: Guided by knowledge graphs, ToG-2 can achieve deep and reliable context retrieval and perform precise graph retrieval by treating documents as node contexts. 2. **Faithful Reasoning**: Through a collaborative retrieval process based on knowledge graphs and texts, ToG-2 uses the retrieved heterogeneous knowledge as the basis for LLMs' reasoning, enhancing the fidelity of the content generated by LLMs. 3. **Efficiency and Effectiveness**: - ToG-2 is a training-free and plug-and-play framework that can be applied to various LLMs. - ToG-2 can operate between any associated knowledge graphs and document databases. For pure document databases, entities can be extracted from documents first through relation extraction or entity co-occurrence, and then graphs can be constructed. - ToG-2 achieves new state-of-the-art (SOTA) performance on multiple complex knowledge reasoning datasets and can enhance the reasoning capabilities of smaller models (e.g., LLAMA-2-13B) to a level comparable to direct reasoning by powerful LLMs (e.g., GPT-3.5). ### Experimental Results Experimental results show that ToG-2 performs excellently on multiple knowledge-intensive datasets, especially in multi-step reasoning tasks. For example, on datasets such as WebQSP, AdvHotpotQA, QALD-10-en, and Zero-Shot RE, ToG-2 significantly outperforms other baseline methods. Additionally, ToG-2 demonstrates a clear advantage in domain-specific reasoning tasks (e.g., ToG-FinQA dataset), further validating its effectiveness in handling complex problems. ### Conclusion By tightly coupling knowledge graph and text retrieval methods, ToG-2 effectively addresses the shortcomings of existing RAG methods in complex reasoning tasks, providing a new solution to enhance the reasoning capabilities and knowledge acquisition of LLMs.

Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation

Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph

Think-on-Graph: Deep and Responsible Reasoning of Large Language Model with Knowledge Graph

Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation

GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoning

Debate on Graph: a Flexible and Reliable Reasoning Framework for Large Language Models

Reasoning on Efficient Knowledge Paths:Knowledge Graph Guides Large Language Model for Domain Question Answering

Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answering

Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models

Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoning

Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning

Decoding on Graphs: Faithful and Sound Reasoning on Knowledge Graphs through Generation of Well-Formed Chains

Retrieval and Reasoning on KGs: Integrate Knowledge Graphs into Large Language Models for Complex Question Answering

Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs

WeKnow-RAG: An Adaptive Approach for Retrieval-Augmented Generation Integrating Web Search and Knowledge Graphs

Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Large Language Models

Logic Query of Thoughts: Guiding Large Language Models to Answer Complex Logic Queries with Knowledge Graphs

GRAG: Graph Retrieval-Augmented Generation

KG-GPT: A General Framework for Reasoning on Knowledge Graphs Using Large Language Models

Clue-Guided Path Exploration: Optimizing Knowledge Graph Retrieval with Large Language Models to Address the Information Black Box Challenge