Abstract:Retrieval-augmented Large Language Models (LLMs) offer substantial benefits in enhancing performance across knowledge-intensive scenarios. However, these methods often face challenges with complex inputs and encounter difficulties due to noisy knowledge retrieval, notably hindering model effectiveness. To address this issue, we introduce BlendFilter, a novel approach that elevates retrieval-augmented LLMs by integrating query generation blending with knowledge filtering. BlendFilter proposes the blending process through its query generation method, which integrates both external and internal knowledge augmentation with the original query, ensuring comprehensive information gathering. Additionally, our distinctive knowledge filtering module capitalizes on the intrinsic capabilities of the LLM, effectively eliminating extraneous data. We conduct extensive experiments on three open-domain question answering benchmarks, and the findings clearly indicate that our innovative BlendFilter surpasses state-of-the-art baselines significantly.

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve This paper aims to address the challenges faced by Retrieval-Augmented Large Language Models (LLMs) when dealing with complex inputs and the issue of noisy information in the retrieved knowledge. #### Specific Problems: 1. **Handling Complex Inputs**: - Current retrieval-augmented methods struggle with complex inputs. Simple queries can effectively identify relevant information, but multifaceted and complex queries may fail to cover all key information, making it difficult to retrieve relevant documents. 2. **Filtering Noisy Knowledge**: - The retrieved knowledge documents may contain irrelevant or misleading information. Typically, the top K documents returned by the retriever are used for augmentation, but these documents may not all be relevant to the task. Including such noisy information in the augmented query can lead to inaccurate outputs from the LLM. ### Solution To address the above issues, the paper proposes **BlendFilter**, a novel framework that enhances the performance of retrieval-augmented LLMs through the integration of Query Generation Blending and Knowledge Filtering. #### Core Components: 1. **Query Generation Blending Module**: - Enriches the original query through various augmentation strategies, forming a composite of queries to tackle the challenges of complex queries. This module combines external and internal knowledge sources for augmentation. 2. **Knowledge Filtering Module**: - Aims to eliminate irrelevant knowledge from the retrieved documents without requiring additional language models, utilizing the LLM's own filtering capabilities. 3. **Answer Generation Module**: - In the final stage, the LLM combines the filtered knowledge with the original query to generate the final answer. ### Main Contributions: 1. **Introduction of a New Query Generation Blending Method**: - Unlike existing works that rely on a single source, this method enriches queries using multiple knowledge sources, thereby covering relevant knowledge more comprehensively. 2. **Proposal of a New Effective Knowledge Filtering Module**: - For the first time, it proposes using the LLM itself as a filter to eliminate irrelevant knowledge. 3. **Extensive Experimental Validation**: - Conducted extensive experiments on three open-domain question-answering benchmarks, showing that the proposed BlendFilter model significantly outperforms baseline models. ### Experimental Results: - **Performance Comparison**: - BlendFilter shows significant performance improvements across different backbone models and datasets. For example, using GPT-3.5-turbo-Instruct, Vicuna 1.5-13b, and Qwen-7b as backbone models, it achieved average improvements of 9.7%, 7.4%, and 14.2%, respectively. - **Generalization Across Different Retrievers**: - BlendFilter performs well with both ColBERT v2 and BM25 retrievers, with particularly notable performance improvements on BM25. - **Retrieval Effectiveness**: - The Knowledge Filtering Module effectively eliminates irrelevant documents, improving retrieval accuracy. - **Importance of Different Queries**: - Original queries, external augmented queries, and internal augmented queries all play important roles in answer generation, especially when using the sparse retriever BM25, where internal augmented queries are particularly crucial. - **Impact of the Number of Retrieved Documents**: - Appropriately increasing the number of retrieved knowledge documents can improve model performance, but too many documents can introduce noisy information, leading to performance degradation. - **Impact of Sampling Temperature**: - Using different sampling temperatures has some impact on model performance, but overall, BlendFilter performs well across different temperature settings.

BlendFilter: Advancing Retrieval-Augmented Large Language Models via Query Generation Blending and Knowledge Filtering

UDKAG: Augmenting Large Vision-Language Models with Up-to-Date Knowledge

Generative Multi-Modal Knowledge Retrieval with Large Language Models

Knowledge-Aware Query Expansion with Large Language Models for Textual and Relational Retrieval

Query Optimization for Parametric Knowledge Refinement in Retrieval-Augmented Large Language Models

InfuserKI: Enhancing Large Language Models with Knowledge Graphs via Infuser-Guided Knowledge Integration

Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy

Learning to Plan for Retrieval-Augmented Large Language Models from Knowledge Graphs

An Interactive Multi-modal Query Answering System with Retrieval-Augmented Large Language Models

Knowledge Fusion of Large Language Models

Advancing Academic Knowledge Retrieval via LLM-enhanced Representation Similarity Fusion

Enhancing Question Answering for Enterprise Knowledge Bases using Large Language Models

CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion

KnowledGPT: Enhancing Large Language Models with Retrieval and Storage Access on Knowledge Bases

Redefining Information Retrieval of Structured Database via Large Language Models

Enhancing Large Language Models with Pseudo- and Multisource- Knowledge Graphs for Open-ended Question Answering

Retrieval-Generation Synergy Augmented Large Language Models

Query Rewriting in Retrieval-Augmented Large Language Models.

FILTER: An Enhanced Fusion Method for Cross-lingual Language Understanding

BIDER: Bridging Knowledge Inconsistency for Efficient Retrieval-Augmented LLMs via Key Supporting Evidence