Abstract:Multi-hop question answering is a challenging task with distinct industrial relevance, and Retrieval-Augmented Generation (RAG) methods based on large language models (LLMs) have become a popular approach to tackle this task. Owing to the potential inability to retrieve all necessary information in a single iteration, a series of iterative RAG methods has been recently developed, showing significant performance improvements. However, existing methods still face two critical challenges: context overload resulting from multiple rounds of retrieval, and over-planning and repetitive planning due to the lack of a recorded retrieval trajectory. In this paper, we propose a novel iterative RAG method called ReSP, equipped with a dual-function summarizer. This summarizer compresses information from retrieved documents, targeting both the overarching question and the current sub-question concurrently. Experimental results on the multi-hop question-answering datasets HotpotQA and 2WikiMultihopQA demonstrate that our method significantly outperforms the state-of-the-art, and exhibits excellent robustness concerning context length.
What problem does this paper attempt to address?
### What problems does this paper attempt to solve?
This paper mainly focuses on two key challenges in the multi - hop question answering task:
1. **Context Overload**:
- During multiple retrieval processes, due to the need to handle longer document contexts, the iterative Retrieval - Augmented Generation (iterative RAG) method is prone to introduce more noise information, increasing the risk of the model missing key information when generating answers.
2. **Over - planning and Repetitive Planning**:
- Existing iterative RAG methods lack the recording of retrieval trajectories, making it difficult for the model to determine whether it has obtained enough information to answer the main question or whether a sub - question has been retrieved. This will cause the model to continue generating new sub - questions when not needed (over - planning), or repeatedly generate sub - questions that have been retrieved (repetitive planning).
To solve these problems, the authors propose a new iterative RAG method - ReSP (Retrieve, Summarize, Plan). ReSP compresses the information extracted from retrieved documents by introducing a dual - function summarizer, and simultaneously summarizes for the main question and the current sub - question. Specifically:
- **Global Evidence Memory**: Used to store summaries of information related to the main question, helping the model determine when to stop iteration.
- **Local Pathway Memory**: Used to store summaries of information related to the current sub - question, preventing repetitive planning.
In this way, ReSP not only solves the problem of context overload but also optimizes the planning process in multi - hop question answering, avoiding over - planning and repetitive planning.
### Experimental Results
The experimental results show that ReSP significantly outperforms existing single - round and iterative RAG methods on two multi - hop question answering datasets, HotpotQA and 2WikiMultihopQA. In particular, on HotpotQA, ReSP improves the F1 score by 4.1 compared to the existing state - of - the - art method (SOTA); on 2WikiMultihopQA, it improves by 5.9.
In addition, ReSP shows good robustness when dealing with contexts of different lengths, and can maintain a stable and concise context in each iteration, thus ensuring that the generated answers are not affected by changes in document length.