Abstract:Revolutionary advancements in Large Language Models have drastically reshaped our interactions with artificial intelligence systems. Despite this, a notable hindrance remains-the deficiency of a long-term memory mechanism within these models. This shortfall becomes increasingly evident in situations demanding sustained interaction, such as personal companion systems and psychological counseling. Therefore, we propose MemoryBank, a novel memory mechanism tailored for LLMs. MemoryBank enables the models to summon relevant memories, continually evolve through continuous memory updates, comprehend, and adapt to a user personality by synthesizing information from past interactions. To mimic anthropomorphic behaviors and selectively preserve memory, MemoryBank incorporates a memory updating mechanism, inspired by the Ebbinghaus Forgetting Curve theory, which permits the AI to forget and reinforce memory based on time elapsed and the relative significance of the memory, thereby offering a human-like memory mechanism. MemoryBank is versatile in accommodating both closed-source models like ChatGPT and open-source models like ChatGLM. We exemplify application of MemoryBank through the creation of an LLM-based chatbot named SiliconFriend in a long-term AI Companion scenario. Further tuned with psychological dialogs, SiliconFriend displays heightened empathy in its interactions. Experiment involves both qualitative analysis with real-world user dialogs and quantitative analysis with simulated dialogs. In the latter, ChatGPT acts as users with diverse characteristics and generates long-term dialog contexts covering a wide array of topics. The results of our analysis reveal that SiliconFriend, equipped with MemoryBank, exhibits a strong capability for long-term companionship as it can provide emphatic response, recall relevant memories and understand user personality.

Topic model for personalized end-to-end task-oriented dialogue

Learning Personalized End-to-End Goal-Oriented Dialog.

Memory-Augmented Dialogue Management for Task-Oriented Dialogue Systems

Personalized Topic Selection Model for Topic-Grounded Dialogue

Task-Oriented Conversation Generation Using Heterogeneous Memory Networks.

Learning to Memorize Entailment and Discourse Relations for Persona-Consistent Dialogues

From easy to hard: Improving personalized response generation of task-oriented dialogue systems by leveraging capacity in open-domain dialogues

Learning to Memorize in Neural Task-Oriented Dialogue Systems

Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots

Topic-Aware Multi-turn Dialogue Modeling

Less is More: Learning to Refine Dialogue History for Personalized Dialogue Generation

A Template-guided Hybrid Pointer Network for Knowledge-basedTask-oriented Dialogue Systems

GraphMemDialog: Optimizing End-to-End Task-Oriented Dialog Systems Using Graph Memory Networks

Dialogue History Matters! Personalized Response Selection in Multi-Turn Retrieval-Based Chatbots

UniMC: A Unified Framework for Long-Term Memory Conversation via Relevance Representation Learning

Personalized Large Language Model Assistant with Evolving Conditional Memory

Know Deeper: Knowledge-Conversation Cyclic Utilization Mechanism for Open-domain Dialogue Generation

A Pre-training Based Personalized Dialogue Generation Model with Persona-sparse Data

Knowledge-based end-to-end memory networks

MemoryBank: Enhancing Large Language Models with Long-Term Memory