Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models

Fei Wang,Xingchen Wan,Ruoxi Sun,Jiefeng Chen,Sercan Ö. Arık

2024-10-10

Abstract:Retrieval-Augmented Generation (RAG), while effective in integrating external knowledge to address the limitations of large language models (LLMs), can be undermined by imperfect retrieval, which may introduce irrelevant, misleading, or even malicious information. Despite its importance, previous studies have rarely explored the behavior of RAG through joint analysis on how errors from imperfect retrieval attribute and propagate, and how potential conflicts arise between the LLMs' internal knowledge and external sources. We find that imperfect retrieval augmentation might be inevitable and quite harmful, through controlled analysis under realistic conditions. We identify the knowledge conflicts between LLM-internal and external knowledge from retrieval as a bottleneck to overcome in the post-retrieval stage of RAG. To render LLMs resilient to imperfect retrieval, we propose Astute RAG, a novel RAG approach that adaptively elicits essential information from LLMs' internal knowledge, iteratively consolidates internal and external knowledge with source-awareness, and finalizes the answer according to information reliability. Our experiments using Gemini and Claude demonstrate that Astute RAG significantly outperforms previous robustness-enhanced RAG methods. Notably, Astute RAG is the only approach that matches or exceeds the performance of LLMs without RAG under worst-case scenarios. Further analysis reveals that Astute RAG effectively resolves knowledge conflicts, improving the reliability and trustworthiness of RAG systems.

Computation and Language,Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

The problem this paper attempts to address is the inaccuracy in responses of large language models (LLMs) when incorporating external knowledge through Retrieval-Augmented Generation (RAG) due to imperfect retrieval results (such as irrelevant, misleading, or even malicious information). Although RAG is highly effective in integrating external knowledge to overcome the inherent knowledge limitations of LLMs, existing research rarely explores the impact of imperfect retrieval on RAG performance and how to handle knowledge conflicts between internal knowledge of LLMs and external sources. Through empirical analysis, the paper finds that imperfect retrieval augmentation is common and harmful under real-world conditions. Therefore, it proposes a new method called Astute RAG, which aims to make LLMs more resilient to imperfections in external retrieval results and effectively resolve conflicts between internal and external knowledge, thereby improving the reliability and credibility of RAG systems.