Abstract:Iterative retrieval refers to the process in which the model continuously queries the retriever during generation to enhance the relevance of the retrieved knowledge, thereby improving the performance of Retrieval-Augmented Generation (RAG). Existing work typically employs few-shot prompting or manually constructed rules to implement iterative retrieval. This introduces additional inference overhead and overlooks the remarkable reasoning capabilities of Large Language Models (LLMs). In this paper, we introduce Auto-RAG, an autonomous iterative retrieval model centered on the LLM's powerful decision-making capabilities. Auto-RAG engages in multi-turn dialogues with the retriever, systematically planning retrievals and refining queries to acquire valuable knowledge. This process continues until sufficient external information is gathered, at which point the results are presented to the user. To this end, we develop a method for autonomously synthesizing reasoning-based decision-making instructions in iterative retrieval and fine-tuned the latest open-source LLMs. The experimental results indicate that Auto-RAG is capable of autonomous iterative interaction with the retriever, effectively leveraging the remarkable reasoning and decision-making abilities of LLMs, which lead to outstanding performance across six benchmarks. Further analysis reveals that Auto-RAG can autonomously adjust the number of iterations based on the difficulty of the questions and the utility of the retrieved knowledge, without requiring any human intervention. Moreover, Auto-RAG expresses the iterative retrieval process in natural language, enhancing interpretability while providing users with a more intuitive experience\footnote{Code is available at \url{<a class="link-external link-https" href="https://github.com/ictnlp/Auto-RAG" rel="external noopener nofollow">this https URL</a>}.

Evaluating Retrieval-Augmented Generation Models for Financial Report Question and Answering

Improving Retrieval for RAG based Question Answering Models on Financial Documents

Multi-Reranker: Maximizing performance of retrieval-augmented generation in the FinanceRAG challenge

Retrieval-Augmented Generation for Large Language Models: A Survey

Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generation

RAG based Question-Answering for Contextual Response Prediction System

Auto-RAG: Autonomous Retrieval-Augmented Generation for Large Language Models

Evaluating Quality of Answers for Retrieval-Augmented Generation: A Strong LLM Is All You Need

Enhancing Q&A with Domain-Specific Fine-Tuning and Iterative Reasoning: A Comparative Study

Towards Optimizing a Retrieval Augmented Generation using Large Language Model on Academic Data

DomainRAG: A Chinese Benchmark for Evaluating Domain-specific Retrieval-Augmented Generation

Rationale-Guided Retrieval Augmented Generation for Medical Question Answering

RAGGED: Towards Informed Design of Retrieval Augmented Generation Systems

Optimizing and Evaluating Enterprise Retrieval-Augmented Generation (RAG): A Content Design Perspective

Towards Understanding Retrieval Accuracy and Prompt Quality in RAG Systems

Know Your RAG: Dataset Taxonomy and Generation Strategies for Evaluating RAG Systems

A Survey on Retrieval-Augmented Text Generation for Large Language Models

Retrieval-Augmented Generation for Domain-Specific Question Answering: A Case Study on Pittsburgh and CMU

Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection

LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering

HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction