Abstract:Revolutionary advancements in Large Language Models have drastically reshaped our interactions with artificial intelligence systems. Despite this, a notable hindrance remains-the deficiency of a long-term memory mechanism within these models. This shortfall becomes increasingly evident in situations demanding sustained interaction, such as personal companion systems and psychological counseling. Therefore, we propose MemoryBank, a novel memory mechanism tailored for LLMs. MemoryBank enables the models to summon relevant memories, continually evolve through continuous memory updates, comprehend, and adapt to a user personality by synthesizing information from past interactions. To mimic anthropomorphic behaviors and selectively preserve memory, MemoryBank incorporates a memory updating mechanism, inspired by the Ebbinghaus Forgetting Curve theory, which permits the AI to forget and reinforce memory based on time elapsed and the relative significance of the memory, thereby offering a human-like memory mechanism. MemoryBank is versatile in accommodating both closed-source models like ChatGPT and open-source models like ChatGLM. We exemplify application of MemoryBank through the creation of an LLM-based chatbot named SiliconFriend in a long-term AI Companion scenario. Further tuned with psychological dialogs, SiliconFriend displays heightened empathy in its interactions. Experiment involves both qualitative analysis with real-world user dialogs and quantitative analysis with simulated dialogs. In the latter, ChatGPT acts as users with diverse characteristics and generates long-term dialog contexts covering a wide array of topics. The results of our analysis reveal that SiliconFriend, equipped with MemoryBank, exhibits a strong capability for long-term companionship as it can provide emphatic response, recall relevant memories and understand user personality.

MEMORYLLM: Towards Self-Updatable Large Language Models

MEMORYLLM: Towards Self-Updatable Large Language Models

Augmenting Language Models with Long-Term Memory

Self-Updatable Large Language Models with Parameter Integration

Enhancing Large Language Model with Self-Controlled Memory Framework

MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory

Unleashing Infinite-Length Input Capacity for Large-scale Language Models with Self-Controlled Memory System.

RET-LLM: Towards a General Read-Write Memory for Large Language Models

Large Language Models Are Semi-Parametric Reinforcement Learning Agents

RecallM: An Adaptable Memory Mechanism with Temporal Understanding for Large Language Models

Large Language Models with Controllable Working Memory

Schrodinger's Memory: Large Language Models

Updating Large Language Models’ Memories with Time Constraints

MemoryBank: Enhancing Large Language Models with Long-Term Memory

$\text{Memory}^3$: Language Modeling with Explicit Memory

CMT: A Memory Compression Method for Continual Knowledge Learning of Large Language Models

Needle in the Haystack for Memory Based Large Language Models

MindLLM: Lightweight Large Language Model Pre-Training, Evaluation and Domain Application

MindLLM: Pre-training Lightweight Large Language Model from Scratch, Evaluations and Domain Applications

Empowering Working Memory for Large Language Model Agents