Compress to Impress: Unleashing the Potential of Compressive Memory in Real-World Long-Term Conversations

Nuo Chen,Hongguang Li,Juhua Huang,Baoyuan Wang,Jia Li

2024-07-01

Abstract:Existing retrieval-based methods have made significant strides in maintaining long-term conversations. However, these approaches face challenges in memory database management and accurate memory retrieval, hindering their efficacy in dynamic, real-world interactions. This study introduces a novel framework, COmpressive Memory-Enhanced Dialogue sYstems (COMEDY), which eschews traditional retrieval modules and memory databases. Instead, COMEDY adopts a "One-for-All" approach, utilizing a single language model to manage memory generation, compression, and response generation. Central to this framework is the concept of compressive memory, which intergrates session-specific summaries, user-bot dynamics, and past events into a concise memory format. To support COMEDY, we curated a large-scale Chinese instruction-tuning dataset, Dolphin, derived from real user-chatbot interactions. Comparative evaluations demonstrate COMEDY's superiority over traditional retrieval-based methods in producing more nuanced and human-like conversational experiences. Our codes are available at <a class="link-external link-https" href="https://github.com/nuochenpku/COMEDY" rel="external noopener nofollow">this https URL</a>.

Computation and Language

What problem does this paper attempt to address?

The main problem this paper attempts to address is the challenge of memory database management and accurate memory retrieval in long-term conversations faced by existing retrieval-based methods. Specifically, these methods perform poorly in dynamic, real-world interactions because they rely on multiple modules (such as memory generators and retrievers) working together, but cannot guarantee the retrieval of relevant and effective memories. Additionally, as conversations accumulate, the scale and complexity of the memory database increase, making management more difficult.

To address these issues, the paper proposes a new framework—**COmpressive Memory-Enhanced Dialogue sYstems (COMEDY)**. The core of the COMEDY framework lies in adopting a "one-stop" approach, utilizing a single language model to manage the entire process from memory generation and compression to final response generation. This framework abandons traditional retrieval modules and memory databases, instead compressing memories to integrate conversation-specific summaries, user-robot interactions, and past events into a concise memory format. This ensures that the generated responses are not only context-aware but also capable of personalized and adaptive adjustments based on changes in the user-robot relationship.

Furthermore, to support the research on COMEDY, the authors collected a large-scale Chinese long-term conversation dataset **Dolphin**, which contains interaction data between real users and chatbots. Through training and evaluation on the Dolphin dataset, COMEDY demonstrates superior performance over traditional retrieval-based methods in generating more detailed and human-like conversational experiences.