Memory GAPS: Would LLMs pass the Tulving Test?

Jean-Marie Chauvet

2024-02-28

Abstract:The Tulving Test was designed to investigate memory performance in recognition and recall tasks. Its results help assess the relevance of the "Synergistic Ecphory Model" of memory and similar RK paradigms in human performance. This paper starts investigating whether the more than forty-year-old framework sheds some light on LLMs' acts of remembering.

Computer Science

What problem does this paper attempt to address?

Memory Traces: Are Transformers Tulving Machines?

Jean-Marie Chauvet

2024-04-12

Abstract:Memory traces--changes in the memory system that result from the perception and encoding of an event--were measured in pioneering studies by Endel Tulving and Michael J. Watkins in 1975. These and further experiments informed the maturation of Tulving's memory model, from the GAPS (General Abstract Processing System} to the SPI (Serial-Parallel Independent) model. Having current top of the line LLMs revisit the original Tulving-Watkins tests may help in assessing whether foundation models completely instantiate or not this class of psychological models.

Artificial Intelligence
Understanding Memorisation in LLMs: Dynamics, Influencing Factors, and Implications

Till Speicher,Mohammad Aflah Khan,Qinyuan Wu,Vedant Nanda,Soumi Das,Bishwamittra Ghosh,Krishna P. Gummadi,Evimaria Terzi

2024-07-27

Abstract:Understanding whether and to what extent large language models (LLMs) have memorised training data has important implications for the reliability of their output and the privacy of their training data. In order to cleanly measure and disentangle memorisation from other phenomena (e.g. in-context learning), we create an experimental framework that is based on repeatedly exposing LLMs to random strings. Our framework allows us to better understand the dynamics, i.e., the behaviour of the model, when repeatedly exposing it to random strings. Using our framework, we make several striking observations: (a) we find consistent phases of the dynamics across families of models (Pythia, Phi and Llama2), (b) we identify factors that make some strings easier to memorise than others, and (c) we identify the role of local prefixes and global context in memorisation. We also show that sequential exposition to different random strings has a significant effect on memorisation. Our results, often surprising, have significant downstream implications in the study and usage of LLMs.

Computation and Language,Machine Learning
Schrodinger's Memory: Large Language Models

Wei Wang,Qing Li

2024-09-27

Abstract:Memory is the foundation of all human activities; without memory, it would be nearly impossible for people to perform any task in daily life. With the development of Large Language Models (LLMs), their language capabilities are becoming increasingly comparable to those of humans. But do LLMs have memory? Based on current performance, LLMs do appear to exhibit memory. So, what is the underlying mechanism of this memory? Previous research has lacked a deep exploration of LLMs' memory capabilities and the underlying theory. In this paper, we use Universal Approximation Theorem (UAT) to explain the memory mechanism in LLMs. We also conduct experiments to verify the memory capabilities of various LLMs, proposing a new method to assess their abilities based on these memory ability. We argue that LLM memory operates like Schrödinger's memory, meaning that it only becomes observable when a specific memory is queried. We can only determine if the model retains a memory based on its output in response to the query; otherwise, it remains indeterminate. Finally, we expand on this concept by comparing the memory capabilities of the human brain and LLMs, highlighting the similarities and differences in their operational mechanisms.

Computation and Language
Think or Remember? Detecting and Directing LLMs Towards Memorization or Generalization

Yi-Fu Fu,Yu-Chieh Tu,Tzu-Ling Cheng,Cheng-Yu Lin,Yi-Ting Yang,Heng-Yi Liu,Keng-Te Liao,Da-Cheng Juan,Shou-De Lin

2024-12-24

Abstract:In this paper, we explore the foundational mechanisms of memorization and generalization in Large Language Models (LLMs), inspired by the functional specialization observed in the human brain. Our investigation serves as a case study leveraging specially designed datasets and experimental-scale LLMs to lay the groundwork for understanding these behaviors. Specifically, we aim to first enable LLMs to exhibit both memorization and generalization by training with the designed dataset, then (a) examine whether LLMs exhibit neuron-level spatial differentiation for memorization and generalization, (b) predict these behaviors using model internal representations, and (c) steer the behaviors through inference-time interventions. Our findings reveal that neuron-wise differentiation of memorization and generalization is observable in LLMs, and targeted interventions can successfully direct their behavior.

Computation and Language
Assessing Episodic Memory in LLMs with Sequence Order Recall Tasks

Mathis Pink,Vy A. Vo,Qinyuan Wu,Jianing Mu,Javier S. Turek,Uri Hasson,Kenneth A. Norman,Sebastian Michelmann,Alexander Huth,Mariya Toneva

2024-10-11

Abstract:Current LLM benchmarks focus on evaluating models' memory of facts and semantic relations, primarily assessing semantic aspects of long-term memory. However, in humans, long-term memory also includes episodic memory, which links memories to their contexts, such as the time and place they occurred. The ability to contextualize memories is crucial for many cognitive tasks and everyday functions. This form of memory has not been evaluated in LLMs with existing benchmarks. To address the gap in evaluating memory in LLMs, we introduce Sequence Order Recall Tasks (SORT), which we adapt from tasks used to study episodic memory in cognitive psychology. SORT requires LLMs to recall the correct order of text segments, and provides a general framework that is both easily extendable and does not require any additional annotations. We present an initial evaluation dataset, Book-SORT, comprising 36k pairs of segments extracted from 9 books recently added to the public domain. Based on a human experiment with 155 participants, we show that humans can recall sequence order based on long-term memory of a book. We find that models can perform the task with high accuracy when relevant text is given in-context during the SORT evaluation. However, when presented with the book text only during training, LLMs' performance on SORT falls short. By allowing to evaluate more aspects of memory, we believe that SORT will aid in the emerging development of memory-augmented models.

Computation and Language,Artificial Intelligence,Machine Learning
A simplified conjoint recognition paradigm for the measurement of gist and verbatim memory.

Christoph Stahl,Karl Christoph Klauer

DOI: https://doi.org/10.1037/0278-7393.34.3.570

2008-01-01

Abstract:The distinction between verbatim and gist memory traces has furthered the understanding of numerous phenomena in various fields, such as false memory research, research on reasoning and decision making, and cognitive development. To measure verbatim and gist memory empirically, an experimental paradigm and multinomial measurement model has been proposed but rarely applied. In the present article, a simplified conjoint recognition paradigm and multinomial model is introduced and validated as a measurement tool for the separate assessment of verbatim and gist memory processes. A Bayesian metacognitive framework is applied to validate guessing processes. Extensions of the model toward incorporating the processes of phantom recollection and erroneous recollection rejection are discussed.
Memory, Consciousness and Large Language Model

Jitang Li,Jinzheng Li

2024-07-07

Abstract:With the development in cognitive science and Large Language Models (LLMs), increasing connections have come to light between these two distinct fields. Building upon these connections, we propose a conjecture suggesting the existence of a duality between LLMs and Tulving's theory of memory. We identify a potential correspondence between Tulving's synergistic ecphory model (SEM) of retrieval and the emergent abilities observed in LLMs, serving as supporting evidence for our conjecture. Furthermore, we speculate that consciousness may be considered a form of emergent ability based on this duality. We also discuss how other theories of consciousness intersect with our research.

Neurons and Cognition,Artificial Intelligence,Computation and Language
On Memorization of Large Language Models in Logical Reasoning

Chulin Xie,Yangsibo Huang,Chiyuan Zhang,Da Yu,Xinyun Chen,Bill Yuchen Lin,Bo Li,Badih Ghazi,Ravi Kumar

2024-10-30

Abstract:Large language models (LLMs) achieve good performance on challenging reasoning benchmarks, yet could also make basic reasoning mistakes. This contrasting behavior is puzzling when it comes to understanding the mechanisms behind LLMs' reasoning capabilities. One hypothesis is that the increasingly high and nearly saturated performance on common reasoning benchmarks could be due to the memorization of similar problems. In this paper, we systematically investigate this hypothesis with a quantitative measurement of memorization in reasoning tasks, using a dynamically generated logical reasoning benchmark based on Knights and Knaves (K&K) puzzles. We found that LLMs could interpolate the training puzzles (achieving near-perfect accuracy) after fine-tuning, yet fail when those puzzles are slightly perturbed, suggesting that the models heavily rely on memorization to solve those training puzzles. On the other hand, we show that while fine-tuning leads to heavy memorization, it also consistently improves generalization performance. In-depth analyses with perturbation tests, cross difficulty-level transferability, probing model internals, and fine-tuning with wrong answers suggest that the LLMs learn to reason on K&K puzzles despite training data memorization. This phenomenon indicates that LLMs exhibit a complex interplay between memorization and genuine reasoning abilities. Finally, our analysis with per-sample memorization score sheds light on how LLMs switch between reasoning and memorization in solving logical puzzles. Our code and data are available at <a class="link-external link-https" href="https://memkklogic.github.io" rel="external noopener nofollow">this https URL</a>.

Computation and Language
Beyond Memorization: The Challenge of Random Memory Access in Language Models

Tongyao Zhu,Qian Liu,Liang Pang,Zhengbao Jiang,Min-Yen Kan,Min Lin

2024-07-22

Abstract:Recent developments in Language Models (LMs) have shown their effectiveness in NLP tasks, particularly in knowledge-intensive tasks. However, the mechanisms underlying knowledge storage and memory access within their parameters remain elusive. In this paper, we investigate whether a generative LM (e.g., GPT-2) is able to access its memory sequentially or randomly. Through carefully-designed synthetic tasks, covering the scenarios of full recitation, selective recitation and grounded question answering, we reveal that LMs manage to sequentially access their memory while encountering challenges in randomly accessing memorized content. We find that techniques including recitation and permutation improve the random memory access capability of LMs. Furthermore, by applying this intervention to realistic scenarios of open-domain question answering, we validate that enhancing random access by recitation leads to notable improvements in question answering. The code to reproduce our experiments can be found at <a class="link-external link-https" href="https://github.com/sail-sg/lm-random-memory-access" rel="external noopener nofollow">this https URL</a>.

Computation and Language,Artificial Intelligence
Exploring anterograde memory: a volumetric MRI study in patients with mild cognitive impairment

N. Philippi,V. Noblet,E. Duron,B. Cretin,C. Boully,I. Wisniewski,M. L. Seux,C. Martin-Hunyadi,E. Chaussade,C. Demuynck,S. Kremer,S. Lehéricy,D. Gounot,J. P. Armspach,O. Hanon,F. Blanc

DOI: https://doi.org/10.1186/s13195-016-0190-1

2016-07-30

Alzheimer's Research and Therapy

Abstract:Abstract Background The aim of this volumetric study was to explore the neuroanatomical correlates of the Free and Cued Selective Reminding Test (FCSRT) and the Delayed Matching-to-Sample—48 items (DMS-48), two tests widely used in France to assess verbal and visual anterograde memory. We wanted to determine to what extent the two tests rely on the medial temporal lobe, and could therefore be predictive of Alzheimer’s disease, in which pathological changes typically start in this region. Methods We analysed data from a cohort of 138 patients with mild cognitive impairment participating in a longitudinal multicentre clinical research study. Verbal memory was assessed using the FCSRT and visual recognition memory was evaluated using the DMS-48. Performances on these two tests were correlated to local grey matter atrophy via structural MRI using voxel-based morphometry. Results Our results confirm the existence of a positive correlation between the volume of the medial temporal lobe and the performance on the FCSRT, prominently on the left, and the performance on the DMS-48, on the right, for the whole group of patients (family-wise error, P < 0.05). Interestingly, this region remained implicated only in the subgroup of patients who had deficient scores on the cued recall of the FCSRT, whereas the free recall was associated with prefrontal aspects. For the DMS-48, it was only implicated for the group of patients whose performances declined between the immediate and delayed trial. Conversely, temporo-parietal cortices were implicated when no decline was observed. Within the medial temporal lobe, the parahippocampal gyrus was prominently involved for the FCSRT and the immediate trial of the DMS-48, whereas the hippocampus was solely involved for the delayed trial of the DMS-48. Conclusions The two tests are able to detect an amnestic profile of the medial temporal type, under the condition that the scores remain deficient after the cued recall of the FCSRT or decline on the delayed recognition trial of the DMS-48. Strategic retrieval as well as perceptual/attentional processes, supported by prefrontal and temporo-parietal cortices, were also found to have an impact on the performances. Finally, the implication of the hippocampus appears time dependent, triggered by a longer delay than the parahippocampus, rather than determined by the sense of recollection or the encoding strength associated with the memory trace.
On the Structural Memory of LLM Agents

Ruihong Zeng,Jinyuan Fang,Siwei Liu,Zaiqiao Meng

2024-12-17

Abstract:Memory plays a pivotal role in enabling large language model~(LLM)-based agents to engage in complex and long-term interactions, such as question answering (QA) and dialogue systems. While various memory modules have been proposed for these tasks, the impact of different memory structures across tasks remains insufficiently explored. This paper investigates how memory structures and memory retrieval methods affect the performance of LLM-based agents. Specifically, we evaluate four types of memory structures, including chunks, knowledge triples, atomic facts, and summaries, along with mixed memory that combines these components. In addition, we evaluate three widely used memory retrieval methods: single-step retrieval, reranking, and iterative retrieval. Extensive experiments conducted across four tasks and six datasets yield the following key insights: (1) Different memory structures offer distinct advantages, enabling them to be tailored to specific tasks; (2) Mixed memory structures demonstrate remarkable resilience in noisy environments; (3) Iterative retrieval consistently outperforms other methods across various scenarios. Our investigation aims to inspire further research into the design of memory systems for LLM-based agents.

Computation and Language,Artificial Intelligence
RecallM: An Adaptable Memory Mechanism with Temporal Understanding for Large Language Models

Brandon Kynoch,Hugo Latapie,Dwane van der Sluis

2023-10-03

Abstract:Large Language Models (LLMs) have made extraordinary progress in the field of Artificial Intelligence and have demonstrated remarkable capabilities across a large variety of tasks and domains. However, as we venture closer to creating Artificial General Intelligence (AGI) systems, we recognize the need to supplement LLMs with long-term memory to overcome the context window limitation and more importantly, to create a foundation for sustained reasoning, cumulative learning and long-term user interaction. In this paper we propose RecallM, a novel architecture for providing LLMs with an adaptable and updatable long-term memory mechanism. Unlike previous methods, the RecallM architecture is particularly effective at belief updating and maintaining a temporal understanding of the knowledge provided to it. We demonstrate through various experiments the effectiveness of this architecture. Furthermore, through our own temporal understanding and belief updating experiments, we show that RecallM is four times more effective than using a vector database for updating knowledge previously stored in long-term memory. We also demonstrate that RecallM shows competitive performance on general question-answering and in-context learning tasks.

Artificial Intelligence,Computation and Language,Symbolic Computation
Quantifying In-Context Reasoning Effects and Memorization Effects in LLMs

Siyu Lou,Yuntian Chen,Xiaodan Liang,Liang Lin,Quanshi Zhang

2024-05-20

Abstract:In this study, we propose an axiomatic system to define and quantify the precise memorization and in-context reasoning effects used by the large language model (LLM) for language generation. These effects are formulated as non-linear interactions between tokens/words encoded by the LLM. Specifically, the axiomatic system enables us to categorize the memorization effects into foundational memorization effects and chaotic memorization effects, and further classify in-context reasoning effects into enhanced inference patterns, eliminated inference patterns, and reversed inference patterns. Besides, the decomposed effects satisfy the sparsity property and the universal matching property, which mathematically guarantee that the LLM's confidence score can be faithfully decomposed into the memorization effects and in-context reasoning effects. Experiments show that the clear disentanglement of memorization effects and in-context reasoning effects enables a straightforward examination of detailed inference patterns encoded by LLMs.

Machine Learning,Artificial Intelligence,Computation and Language,Computer Vision and Pattern Recognition
[The Brumory test, an incidental long-term memory task designed for foreign, non-French-speaking people with low educational level]

V Vanderaspoilden,D Nury,J Frisque,P Peigneux

DOI: https://doi.org/10.1016/j.neurol.2015.08.003

Abstract:Cognitive assessment among foreign patients is a growing need for several reasons: foreign patients have a different culture, they have an insufficient command of the language of the consulting center, and the available cognitive tools are largely unsuitable. For these reasons, we developed a non-verbal test of long-term memory called the Brumory test. This test is based on incident encoding of 48 colored images followed by retrieval by recognition. We compared the performance of indigenous participants with that of immigrant participants (mainly from Morocco). Immigrant participants did not speak French properly and had a low educational level. The results indicate no significant difference in memory performance between the two groups of participants. Moreover, the instructions were easily understood by immigrant participants, despite the fact they do not master French. We conclude that the Brumory test is an appropriate test to assess memory among foreign non-French-speaking patients people with low educational level.
Large-scale study of human memory for meaningful narratives

Antonios Georgiou,Tankut Can,Mikhail Katkov,Misha Tsodyks

2024-11-23

Abstract:The statistical study of human memory requires large-scale experiments, involving many stimuli conditions and test subjects. While this approach has proven to be quite fruitful for meaningless material such as random lists of words, naturalistic stimuli, like narratives, have until now resisted such a large-scale study, due to the quantity of manual labor required to design and analyze such experiments. In this work, we develop a pipeline that uses large language models (LLMs) both to design naturalistic narrative stimuli for large-scale recall and recognition memory experiments, as well as to analyze the results. We performed online memory experiments with a large number of participants and collected recognition and recall data for narratives of different sizes. We found that both recall and recognition performance scale linearly with narrative length; however, for longer narratives people tend to summarize the content rather than recalling precise details. To investigate the role of narrative comprehension in memory, we repeated these experiments using scrambled versions of the narratives. Although recall performance declined significantly, recognition remained largely unaffected. Recalls in this condition seem to follow the original narrative order rather than the actual scrambled presentation, pointing to a contextual reconstruction of the story in memory. Finally, using LLM text embeddings, we construct a simple measure for each clause based on semantic similarity to the whole narrative, that shows a strong correlation with recall probability. Overall, our work demonstrates the power of LLMs in accessing new regimes in the study of human memory, as well as suggesting novel psychologically informed benchmarks for LLM performance.

Computation and Language,Neurons and Cognition
The impact of working memory testing on long-term associative memory

Kathy Y. Xie,Patricia A. Reuter-Lorenz

DOI: https://doi.org/10.3758/s13421-024-01568-x

2024-05-08

Memory & Cognition

Abstract:The long-term fate of to-be-remembered information depends in part on the conditions of initial learning, including mental operations engaged via working memory. However, the mechanistic role of working memory (WM) processes in subsequent episodic memory (EM) remains unclear. Does re-exposure to word-pairs during WM recognition testing improve EM for those associations? Are benefits from WM re-exposure greater after an opportunity for retrieval practice compared to mere re-exposure to the memoranda? These questions are addressed in three experiments (N = 460) designed to assess whether WM-based recognition testing benefits long-term associative memory relative to WM-based restudying. Our results show null or negative benefits of WM recognition testing minutes later when initial WM accuracy was not considered. An EM benefit of WM recognition testing only emerges when the analyses are limited to pairs responded to correctly during WM. However, even when compared with accurate WM recognition, restudying can lead to similar associative EM benefits in specific experimental conditions. Taken together, the present results suggest that while WM re-exposure to studied pairs is beneficial to long-term associative memory, successful retrieval on initial tests may be a necessary but insufficient condition for the emergence of a "WM-based testing effect." We consider these results in relation to several hypotheses proposed to explain the testing effect in long-term memory (LTM). In view of empirical parallels with the LTM testing effect, we propose that similar processes influence the benefits of practice tests administered within the canonical boundaries of WM, suggesting continuities in memory over the short and long term.

psychology, experimental
Memorization Over Reasoning? Exposing and Mitigating Verbatim Memorization in Large Language Models' Character Understanding Evaluation

Yuxuan Jiang,Francis Ferraro

2024-12-19

Abstract:Recently, Large Language Models (LLMs) have shown impressive performance in character understanding tasks, such as analyzing the roles, personalities, and relationships of fictional characters. However, the extensive pre-training corpora used by LLMs raise concerns that they may rely on memorizing popular fictional works rather than genuinely understanding and reasoning about them. In this work, we argue that 'gist memory'-capturing essential meaning - should be the primary mechanism for character understanding tasks, as opposed to 'verbatim memory' - exact match of a string. We introduce a simple yet effective method to mitigate mechanized memorization in character understanding evaluations while preserving the essential implicit cues needed for comprehension and reasoning. Our approach reduces memorization-driven performance on popular fictional works from 96% accuracy to 72% and results in up to an 18% drop in accuracy across various character understanding tasks. These findings underscore the issue of data contamination in existing benchmarks, which often measure memorization rather than true character understanding.

Computation and Language
Word Meaning Contributes to Free Recall Performance in Supraspan Verbal List-Learning Tests

Sandrine Cremona,Gaël Jobard,Laure Zago,Emmanuel Mellet

DOI: https://doi.org/10.3389/fpsyg.2020.02043

IF: 3.8

2020-08-14

Frontiers in Psychology

Abstract:Supraspan verbal list-learning tests, such as the Rey Auditory Verbal Learning Test (RAVLT), are classic neuropsychological tests for assessing verbal memory. In this study, we investigated the impact of the meaning of the words to be learned on three memory stages [short-term recall (STR), learning, and delayed recall (DR)] in a cohort of 447 healthy adults. First, we compared scores obtained from the RAVLT (word condition) to those of an alternative version of this test using phonologically similar but meaningless items (pseudoword condition) and observed how each score varied as a function of age and sex. Then, we collected the participants' self-reported strategies to retain the word and pseudoword lists and examined if these strategies mediated the age and sex effects on memory scores. The word condition resulted in higher memory scores than pseudoword condition at each memory stage and even canceled out, for the learning stage, the detrimental effect of age that was observed for the short-term and DR. When taking sex into account, the word advantage was observed only in women for STR. The self-reported strategies, which were similar for words and pseudowords, were based on the position of the item on the list (word: 53%, pseudoword: 37%) or the meaning of the item (word: 64%, pseudoword: 58%) and were used alone or in combination. The best memory performance was associated with the meaning strategy in the word condition and with the combination of the meaning and position strategies in the pseudoword condition. Finally, we found that the word advantage observed in women for STR was mediated by the use of the meaning strategy. The RAVLT scores were thus highly dependent on word meaning, notably because it allowed efficient semantic knowledge-based strategies. Within the framework of Tulving's declarative memory model, these results are at odds with the depiction of the RAVLT as a verbal <i>episodic memory</i> test as it is increasingly referred to in the literature.

psychology, multidisciplinary
Neural evidence for the use of digit-image mnemonic in a superior memorist: an fMRI study.

Li-Jun Yin,Yu-Ting Lou,Ming-Xia Fan,Zhao-Xin Wang,Yi Hu

DOI: https://doi.org/10.3389/fnhum.2015.00109

IF: 3.473

2015-01-01

Frontiers in Human Neuroscience

Abstract:Some superior memorists demonstrated exceptional memory for reciting a large body of information. The underlying neural correlates, however, are seldom addressed. C.L., the current holder of Guinness World Record for reciting 67,890 digits in π, participated in this functional magnetic resonance imaging (fMRI) study. Thirteen participants without any mnemonics training were included as controls. Our previous studies suggested that C.L. used a digit-image mnemonic in studying and recalling lists of digits, namely associating 2-digit groups of "00" to "99" with images and generating vivid stories out of them (Hu et al., 2009). Thus, 2-digit condition was included, with 1-digit numbers and letters as control conditions. We hypothesized that 2-digit condition in C.L. should elicit the strongest activity in the brain regions which are associated with his mnemonic. Functional MRI results revealed that bilateral frontal poles (FPs, BA10), left superior parietal lobule (SPL), left premotor cortex (PMC), and left dorsolateral prefrontal cortex (DLPFC), were more engaged in both the study and recall phase of 2-digit condition for C.L. relative to controls. Moreover, the left middle/inferior frontal gyri (M/IFG) and intraparietal sulci (IPS) were less engaged in the study phase of 2-digit condition for C.L. (vs. controls). These results suggested that C.L. relied more on brain regions that are associated with episodic memory other than verbal rehearsal while he used his mnemonic strategies. This study supported theoretical accounts of restructured cognitive mechanisms for the acquisition of superior memory performance.

Memory GAPS: Would LLMs pass the Tulving Test?

Memory Traces: Are Transformers Tulving Machines?

Understanding Memorisation in LLMs: Dynamics, Influencing Factors, and Implications

Schrodinger's Memory: Large Language Models

Think or Remember? Detecting and Directing LLMs Towards Memorization or Generalization

Assessing Episodic Memory in LLMs with Sequence Order Recall Tasks

A simplified conjoint recognition paradigm for the measurement of gist and verbatim memory.

Memory, Consciousness and Large Language Model

On Memorization of Large Language Models in Logical Reasoning

Beyond Memorization: The Challenge of Random Memory Access in Language Models

Exploring anterograde memory: a volumetric MRI study in patients with mild cognitive impairment

On the Structural Memory of LLM Agents

RecallM: An Adaptable Memory Mechanism with Temporal Understanding for Large Language Models

Quantifying In-Context Reasoning Effects and Memorization Effects in LLMs

[The Brumory test, an incidental long-term memory task designed for foreign, non-French-speaking people with low educational level]

Large-scale study of human memory for meaningful narratives

The impact of working memory testing on long-term associative memory

Memorization Over Reasoning? Exposing and Mitigating Verbatim Memorization in Large Language Models' Character Understanding Evaluation

Word Meaning Contributes to Free Recall Performance in Supraspan Verbal List-Learning Tests

Neural evidence for the use of digit-image mnemonic in a superior memorist: an fMRI study.