TRACE the Evidence: Constructing Knowledge-Grounded Reasoning Chains for Retrieval-Augmented Generation

Jinyuan Fang,Zaiqiao Meng,Craig Macdonald

2024-06-17

Abstract:Retrieval-augmented generation (RAG) offers an effective approach for addressing question answering (QA) tasks. However, the imperfections of the retrievers in RAG models often result in the retrieval of irrelevant information, which could introduce noises and degrade the performance, especially when handling multi-hop questions that require multiple steps of reasoning. To enhance the multi-hop reasoning ability of RAG models, we propose TRACE. TRACE constructs knowledge-grounded reasoning chains, which are a series of logically connected knowledge triples, to identify and integrate supporting evidence from the retrieved documents for answering questions. Specifically, TRACE employs a KG Generator to create a knowledge graph (KG) from the retrieved documents, and then uses an Autoregressive Reasoning Chain Constructor to build reasoning chains. Experimental results on three multi-hop QA datasets show that TRACE achieves an average performance improvement of up to 14.03% compared to using all the retrieved documents. Moreover, the results indicate that using reasoning chains as context, rather than the entire documents, is often sufficient to correctly answer questions.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in multi - hop question - answering tasks, the Retrieval - Augmented Generation (RAG) model often introduces irrelevant information due to the imperfection of the retriever, thus affecting the performance of the model. Specifically, when dealing with multi - hop questions that require multiple reasoning steps to obtain the correct answer, the retrieved irrelevant documents will introduce noise, mislead the model, and reduce its reasoning ability. In order to improve the performance of the RAG model in multi - hop reasoning tasks, the author proposes the TRACE method. By constructing an inference chain supported by a knowledge graph to identify and integrate the supporting evidence in the documents, the influence of irrelevant information is reduced and the reasoning ability of the model is improved.

TRACE the Evidence: Constructing Knowledge-Grounded Reasoning Chains for Retrieval-Augmented Generation

Exploiting Reasoning Chains for Multi-hop Science Question Answering

Exploiting Reasoning Chains for Multi-hop Science Question Answering.

Improving embedded knowledge graph multi-hop question answering by introducing relational chain reasoning

Towards Graph-hop Retrieval and Reasoning in Complex Question Answering over Textual Database

Graph Reasoning for Question Answering with Triplet Retrieval

Reasoning Path Generation for Answering Multi-hop Questions over Knowledge Graph

Multi-hop Question Answering via Reasoning Chains

Hierarchical Retrieval-Augmented Generation Model with Rethink for Multi-hop Question Answering

Leveraging Structured Information for Explainable Multi-hop Question Answering and Reasoning

DR-RAG: Applying Dynamic Document Relevance to Retrieval-Augmented Generation for Question-Answering

A Hybrid RAG System with Comprehensive Enhancement on Complex Reasoning

Multi-hop question answering over incomplete knowledge graph with abstract conceptual evidence

ReasonChainQA: Text-based Complex Question Answering with Explainable Evidence Chains

Enhancing Complex Question Answering over Knowledge Graphs through Evidence Pattern Retrieval

Retrieving, Rethinking and Revising: The Chain-of-Verification Can Improve Retrieval Augmented Generation

RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement

RPR-KGQA: Relational Path Reasoning for Multi-hop Question Answering with Knowledge Graph

Answering Questions by Meta-Reasoning over Multiple Chains of Thought

CoTKR: Chain-of-Thought Enhanced Knowledge Rewriting for Complex Knowledge Graph Question Answering