Ever-Evolving Memory by Blending and Refining the Past

Seo Hyun Kim,Keummin Ka,Yohan Jo,Seung-won Hwang,Dongha Lee,Jinyoung Yeo

2024-04-07

Abstract:For a human-like chatbot, constructing a long-term memory is crucial. However, current large language models often lack this capability, leading to instances of missing important user information or redundantly asking for the same information, thereby diminishing conversation quality. To effectively construct memory, it is crucial to seamlessly connect past and present information, while also possessing the ability to forget obstructive information. To address these challenges, we propose CREEM, a novel memory system for long-term conversation. Improving upon existing approaches that construct memory based solely on current sessions, CREEM blends past memories during memory formation. Additionally, we introduce a refining process to handle redundant or outdated information. Unlike traditional paradigms, we view responding and memory construction as inseparable tasks. The blending process, which creates new memories, also serves as a reasoning step for response generation by informing the connection between past and present. Through evaluation, we demonstrate that CREEM enhances both memory and response qualities in multi-session personalized dialogues.

Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to construct a memory system that can effectively integrate past and current information in long - term conversations while having the ability to forget obsolete or redundant information, so as to improve the conversation quality of chatbots. Specifically, in view of the current lack of long - term memory in large - language models (LLMs), the paper proposes a novel memory framework named CREEM, which aims to construct and maintain long - term memory through the mixing and refining processes, thereby enhancing the memory quality and response quality in multi - session personalized conversations. The paper points out that although current large - language models have promoted interaction with humans, they often lack long - term memory, resulting in situations such as forgetting important information or repeatedly asking for the same information, which reduces the quality of conversations. CREEM solves these problems in the following ways: - **Blend**: Consider past memories when forming new memories to ensure coherence and relevance. - **Refine**: When adding new memories, refine past memories to maintain overall consistency and handle redundant or obsolete information. In addition, the paper also verifies the effectiveness of CREEM through experiments, including memory evaluation, contradiction level evaluation, question - answering evaluation, and response generation evaluation, etc., proving the superior performance of CREEM in improving memory quality and response quality.

Ever-Evolving Memory by Blending and Refining the Past

MemoryBank: Enhancing Large Language Models with Long-Term Memory

Dual Memory Model for Experience-Once Task-Incremental Lifelong Learning.

Commonsense-augmented Memory Construction and Management in Long-term Conversations via Context-aware Persona Refinement

LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory

Mixed-Session Conversation with Egocentric Memory

A Novel Linguistic-Aware Memory Structure for Enhancing the Response Generation

Memoria: Resolving Fateful Forgetting Problem through Human-Inspired Memory Architecture

CLFR-M: Continual Learning Framework for Robots Via Human Feedback and Dynamic Memory

MemBench: Towards Real-world Evaluation of Memory-Augmented Dialogue Systems

Towards Lifelong Dialogue Agents via Relation-aware Memory Construction and Timeline-augmented Response Generation

Personalized Large Language Model Assistant with Evolving Conditional Memory

"My agent understands me better": Integrating Dynamic Human-like Memory Recall and Consolidation in LLM-Based Agents

Compress to Impress: Unleashing the Potential of Compressive Memory in Real-World Long-Term Conversations

UniMC: A Unified Framework for Long-Term Memory Conversation via Relevance Representation Learning

Response-Anticipated Memory for On-Demand Knowledge Integration in Response Generation

Continual Memory: Can We Reason after Long-Term Memorization?

Evolving Large Language Model Assistant with Long-Term Conditional Memory

RAM: Towards an Ever-Improving Memory System by Learning from Communications

A Machine with Short-Term, Episodic, and Semantic Memory Systems

Integrating Temporal Representations for Dynamic Memory Retrieval and Management in Large Language Models