Abstract:In order to thrive in hostile and ever-changing natural environments, mammalian brains evolved to store large amounts of knowledge about the world and continually integrate new information while avoiding catastrophic forgetting. Despite the impressive accomplishments, large language models (LLMs), even with retrieval-augmented generation (RAG), still struggle to efficiently and effectively integrate a large amount of new experiences after pre-training. In this work, we introduce HippoRAG, a novel retrieval framework inspired by the hippocampal indexing theory of human long-term memory to enable deeper and more efficient knowledge integration over new experiences. HippoRAG synergistically orchestrates LLMs, knowledge graphs, and the Personalized PageRank algorithm to mimic the different roles of neocortex and hippocampus in human memory. We compare HippoRAG with existing RAG methods on multi-hop question answering and show that our method outperforms the state-of-the-art methods remarkably, by up to 20%. Single-step retrieval with HippoRAG achieves comparable or better performance than iterative retrieval like IRCoT while being 10-30 times cheaper and 6-13 times faster, and integrating HippoRAG into IRCoT brings further substantial gains. Finally, we show that our method can tackle new types of scenarios that are out of reach of existing methods. Code and data are available at

What problem does this paper attempt to address?

This paper presents a new framework called HippoRAG, inspired by the theory of long-term memory in the human hippocampus, aiming to address the efficiency and effectiveness issues of large-scale language models (LLMs) in integrating vast amounts of new experiences. Although existing retrieval-augmented generation (RAG) methods have partially addressed this problem, they still struggle to effectively integrate new knowledge across paragraphs. HippoRAG combines LLMs, knowledge graphs, and personalized PageRank algorithm to simulate the different roles of the human neocortex and hippocampus in memory, achieving deeper and more efficient knowledge integration. When dealing with tasks that require knowledge integration, such as multi-hop question answering, HippoRAG outperforms existing RAG methods by improving performance by about 3% to 20%, and its single-step retrieval is faster and more cost-effective than iterative retrieval methods. The paper mentions that existing RAG methods perform poorly in tasks that require integration of information across paragraphs due to independent encoding of each new paragraph. However, HippoRAG can quickly find relevant subgraphs and perform multi-hop reasoning by establishing and utilizing association graphs, thereby achieving knowledge integration in single-step retrieval. Experimental results show that HippoRAG performs better than current methods on two multi-hop question answering datasets, and its performance further improves when combined with multi-step retrieval methods like IRCoT. Additionally, HippoRAG can handle path-finding multi-hop question answering problems that other methods currently cannot address. In conclusion, this paper aims to improve the memory capability of large-scale language models by imitating the human memory mechanism, thereby enhancing their performance in handling complex knowledge integration tasks.

HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models

ActiveRAG: Autonomously Knowledge Assimilation and Accommodation through Retrieval-Augmented Agents

Toward Optimal Search and Retrieval for RAG

IM-RAG: Multi-Round Retrieval-Augmented Generation Through Learning Inner Monologues

Retrieval-Augmented Generation for Large Language Models: A Survey

Enhancing LLM Intelligence with ARM-RAG: Auxiliary Rationale Memory for Retrieval Augmented Generation

Open-RAG: Enhanced Retrieval-Augmented Reasoning with Open-Source Large Language Models

Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models

RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation

Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG

RAG-Modulo: Solving Sequential Tasks using Experience, Critics, and Language Models

AssistRAG: Boosting the Potential of Large Language Models with an Intelligent Information Assistant

Auto-RAG: Autonomous Retrieval-Augmented Generation for Large Language Models

MemoRAG: Moving towards Next-Gen RAG Via Memory-Inspired Knowledge Discovery

Meta Knowledge for Retrieval Augmented Large Language Models

Empowering Large Language Models to Set up a Knowledge Retrieval Indexer via Self-Learning

OG-RAG: Ontology-Grounded Retrieval-Augmented Generation For Large Language Models

Retriever-and-Memory: Towards Adaptive Note-Enhanced Retrieval-Augmented Generation

Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generation

M-RAG: Reinforcing Large Language Model Performance through Retrieval-Augmented Generation with Multiple Partitions

Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation